[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:42 dist-test-slave-7f01 krb5kdc[22789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:42 dist-test-slave-7f01 krb5kdc[22789](info): set up 2 sockets
Aug 06 01:06:42 dist-test-slave-7f01 krb5kdc[22789](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 01:06:44.140669 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:06:44 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:06:44.357080 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40883
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:44.701642 22813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:44.701927 22813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:44.702090 22813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:44.709548 22813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:44.709662 22813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:44.709777 22813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:44.709884 22813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:44.709964 22813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:44.720142 22813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:44.721566 22813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:44.722960 22813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:44.736052 22820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:44.736011 22824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:44.735963 22827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:44.736835 22813 server_base.cc:1034] running on GCE node
I20260806 01:06:44.737689 22813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:44.739184 22813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:44.740418 22813 hybrid_clock.cc:648] HybridClock initialized: now 1785978404740361 us; error 46 us; skew 500 ppm
I20260806 01:06:44.740772 22813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:44 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978404, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:44.744201 22813 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:06:44.745800 22813 webserver.cc:460] Webserver started at http://127.22.63.126:39151/ using document root <none> and password file <none>
I20260806 01:06:44.746500 22813 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:44.746598 22813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:44.746876 22813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:44.749953 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "ab0a1f1014d04843a9e3a0ce220952c8"
format_stamp: "Formatted at 2026-08-06 01:06:44 on dist-test-slave-7f01"
server_key: "8d209a11df9cc3e6b2d5b330036a5e8c"
server_key_iv: "ed8eca3c1737bc256e63d1a387692679"
server_key_version: "encryptionkey@0"
I20260806 01:06:44.750689 22813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "ab0a1f1014d04843a9e3a0ce220952c8"
format_stamp: "Formatted at 2026-08-06 01:06:44 on dist-test-slave-7f01"
server_key: "8d209a11df9cc3e6b2d5b330036a5e8c"
server_key_iv: "ed8eca3c1737bc256e63d1a387692679"
server_key_version: "encryptionkey@0"
I20260806 01:06:44.756016 22813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:06:44.759508 22836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:44.760949 22813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:44.761101 22813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "ab0a1f1014d04843a9e3a0ce220952c8"
format_stamp: "Formatted at 2026-08-06 01:06:44 on dist-test-slave-7f01"
server_key: "8d209a11df9cc3e6b2d5b330036a5e8c"
server_key_iv: "ed8eca3c1737bc256e63d1a387692679"
server_key_version: "encryptionkey@0"
I20260806 01:06:44.761268 22813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:44.777441 22813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:44.782066 22813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:44.782336 22813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:44.801589 22813 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40883
I20260806 01:06:44.801609 22905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40883 every 8 connection(s)
I20260806 01:06:44.803030 22813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:44.806905 22907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:44.809005 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22813
I20260806 01:06:44.809365 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:06:44.811941 22781 external_mini_cluster.cc:1428] Setting key a70ab03bf5b6e9cc98ff991a294074a6
I20260806 01:06:44.820559 22907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8: Bootstrap starting.
I20260806 01:06:44.823518 22907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:44.824599 22907 log.cc:824] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:44.828039 22907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8: No bootstrap required, opened a new log
I20260806 01:06:44.838039 22907 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } }
I20260806 01:06:44.838551 22907 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:44.838670 22907 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab0a1f1014d04843a9e3a0ce220952c8, State: Initialized, Role: FOLLOWER
I20260806 01:06:44.839311 22907 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } }
I20260806 01:06:44.839524 22907 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:44.839707 22907 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:44.840049 22907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:44.842437 22907 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } }
I20260806 01:06:44.842963 22907 leader_election.cc:302] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab0a1f1014d04843a9e3a0ce220952c8; no voters:
I20260806 01:06:44.843621 22907 leader_election.cc:288] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:44.843773 22912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 1 FOLLOWER]: Leader election won for term 1
Aug 06 01:06:44 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:44.844538 22912 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [term 1 LEADER]: Becoming Leader. State: Replica: ab0a1f1014d04843a9e3a0ce220952c8, State: Running, Role: LEADER
I20260806 01:06:44.845208 22912 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } }
I20260806 01:06:44.845991 22907 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:44.848312 22913 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab0a1f1014d04843a9e3a0ce220952c8. Latest consensus state: current_term: 1 leader_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } } }
I20260806 01:06:44.848548 22914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab0a1f1014d04843a9e3a0ce220952c8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40883 } } }
I20260806 01:06:44.848795 22913 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:44.848805 22914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:44.861317 22918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:44.861806 22909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:44.816746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.22.63.126:40883)
0806 01:06:44.817802 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:44.817817 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:44.819243 (+ 1426us) server_negotiation.cc:407] Connection header received
0806 01:06:44.828818 (+ 9575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:44.828848 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:44.829346 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:44.829782 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:44.832802 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:44.834936 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:44.837260 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:44.838310 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:44.838522 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:44.846010 (+ 7488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:44.846034 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:44.846054 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:44.846114 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:44.851895 (+ 5781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:44.853276 (+ 1381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:44.853290 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:44.853312 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:44.853588 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:44.854411 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:44.854421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:44.854427 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:44.855489 (+ 1062us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:44.855925 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:44.859376 (+ 3451us) server_negotiation.cc:299] Negotiation successful
0806 01:06:44.859938 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":452,"threads_started":1}
I20260806 01:06:44.871488 22918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:44.881587 22918 catalog_manager.cc:1347] Generated new cluster ID: aa58a818327f440b9f4812479fa3be41
I20260806 01:06:44.881767 22918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:44.892802 22918 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:44.894668 22918 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:44.901674 22918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab0a1f1014d04843a9e3a0ce220952c8: Generated new TSK 0
I20260806 01:06:44.902697 22918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:06:44.964790 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:45.364253 22939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:45.364589 22939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:45.364748 22939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:45.372400 22939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:45.372510 22939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:45.372627 22939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:06:45.382829 22939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:45.384223 22939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:45.385591 22939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:45.400298 22944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:45.400265 22945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:45.400360 22950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:45.401691 22939 server_base.cc:1034] running on GCE node
I20260806 01:06:45.402288 22939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:45.403208 22939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:45.404412 22939 hybrid_clock.cc:648] HybridClock initialized: now 1785978405404373 us; error 27 us; skew 500 ppm
I20260806 01:06:45.404749 22939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:45 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978405, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:45.413205 22939 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:06:45.414811 22939 webserver.cc:460] Webserver started at http://127.22.63.65:35485/ using document root <none> and password file <none>
I20260806 01:06:45.415560 22939 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:45.415716 22939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:45.416096 22939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:45.419112 22939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "b5b7b20461b54796b319217f6e3bedb0"
format_stamp: "Formatted at 2026-08-06 01:06:45 on dist-test-slave-7f01"
server_key: "14f9ee058a20561083966452edf24e8c"
server_key_iv: "e5b9a2feae9d49275a439f27cb5f0d6b"
server_key_version: "encryptionkey@0"
I20260806 01:06:45.420001 22939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "b5b7b20461b54796b319217f6e3bedb0"
format_stamp: "Formatted at 2026-08-06 01:06:45 on dist-test-slave-7f01"
server_key: "14f9ee058a20561083966452edf24e8c"
server_key_iv: "e5b9a2feae9d49275a439f27cb5f0d6b"
server_key_version: "encryptionkey@0"
I20260806 01:06:45.425422 22939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 01:06:45.429061 22961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:45.430549 22939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:06:45.430743 22939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "b5b7b20461b54796b319217f6e3bedb0"
format_stamp: "Formatted at 2026-08-06 01:06:45 on dist-test-slave-7f01"
server_key: "14f9ee058a20561083966452edf24e8c"
server_key_iv: "e5b9a2feae9d49275a439f27cb5f0d6b"
server_key_version: "encryptionkey@0"
I20260806 01:06:45.430963 22939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:45.455603 22939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:45.460212 22939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:45.460541 22939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:45.461565 22939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:45.463070 22939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:45.463172 22939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:45.463325 22939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:45.463395 22939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:45.499358 22939 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:38479
I20260806 01:06:45.499415 23128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:38479 every 8 connection(s)
I20260806 01:06:45.500793 22939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:06:45.511029 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22939
I20260806 01:06:45.511297 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:06:45.511998 22781 external_mini_cluster.cc:1428] Setting key 3ed3c42fa00a7c3aa9bc4e78c7d864a6
Aug 06 01:06:45 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978405, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:45.521528 23132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:45.503655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:46485 (local address 127.22.63.126:40883)
0806 01:06:45.504205 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:45.504212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:45.505319 (+ 1107us) server_negotiation.cc:407] Connection header received
0806 01:06:45.507201 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:45.507207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:45.507279 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:45.507400 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:45.508970 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:45.509725 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:45.511253 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:45.511728 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:45.511927 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:45.515693 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:45.515698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:45.515701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:45.515733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:45.518379 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:45.519127 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:45.519133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:45.519136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:45.519260 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:45.519703 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:45.519709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:45.519712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:45.520032 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:45.520194 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:45.520912 (+ 718us) server_negotiation.cc:299] Negotiation successful
0806 01:06:45.521142 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":285,"threads_started":1}
I20260806 01:06:45.522642 23134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:45.504368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40883 (local address 127.22.63.65:46485)
0806 01:06:45.505132 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:45.505183 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:45.506905 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:45.507599 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:45.507607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:45.508233 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:45.508801 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:45.508818 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:45.509853 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:45.509857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:45.511092 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:45.511105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:45.512042 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:45.512049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:45.512290 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:45.513008 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:45.513032 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:45.515464 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:45.518523 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:45.518533 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:45.518549 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:45.518991 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:45.519378 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:45.519382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:45.519386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:45.519578 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:45.520173 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:45.520181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:45.520530 (+ 349us) client_negotiation.cc:769] Sending connection context
0806 01:06:45.520817 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 01:06:45.521372 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":308,"threads_started":1}
I20260806 01:06:45.524380 23129 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40883
I20260806 01:06:45.524703 23129 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:45.525475 23129 heartbeater.cc:511] Master 127.22.63.126:40883 requested a full tablet report, sending...
I20260806 01:06:45.527696 22864 ts_manager.cc:194] Registered new tserver with Master: b5b7b20461b54796b319217f6e3bedb0 (127.22.63.65:38479)
I20260806 01:06:45.529273 22864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:46485
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:06:45.563115 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:45.963457 23139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:45.963989 23139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:45.964179 23139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:45.976459 23139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:45.976681 23139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:45.976886 23139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:06:45.991083 23139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:45.992842 23139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:45.994705 23139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:46.014835 23158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:46.014906 23149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:46.015293 23152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:46.017658 23139 server_base.cc:1034] running on GCE node
I20260806 01:06:46.018471 23139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:46.019531 23139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:46.020788 23139 hybrid_clock.cc:648] HybridClock initialized: now 1785978406020739 us; error 41 us; skew 500 ppm
I20260806 01:06:46.021147 23139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:46.024477 23139 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:06:46.026247 23139 webserver.cc:460] Webserver started at http://127.22.63.66:46089/ using document root <none> and password file <none>
I20260806 01:06:46.027002 23139 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:46.027153 23139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:46.027482 23139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:46.030544 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "22e876f906bc4cb69eea9f8e28833860"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "b0f50664d8ca4cc205589449edfb85e9"
server_key_iv: "81fc3c119a03aacfd9f684958b904a92"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.031401 23139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "22e876f906bc4cb69eea9f8e28833860"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "b0f50664d8ca4cc205589449edfb85e9"
server_key_iv: "81fc3c119a03aacfd9f684958b904a92"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.037981 23139 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260806 01:06:46.042328 23169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.043985 23139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:06:46.044206 23139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "22e876f906bc4cb69eea9f8e28833860"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "b0f50664d8ca4cc205589449edfb85e9"
server_key_iv: "81fc3c119a03aacfd9f684958b904a92"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.044467 23139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:46.082221 23139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:46.087867 23139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:46.088236 23139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:46.089529 23139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:46.091152 23139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:46.091276 23139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.091463 23139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:46.091552 23139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.133407 23139 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:37757
I20260806 01:06:46.133474 23333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:37757 every 8 connection(s)
I20260806 01:06:46.135030 23139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:06:46.138931 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23139
I20260806 01:06:46.139155 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:06:46.139645 22781 external_mini_cluster.cc:1428] Setting key 9adf2c4ef2e066e82f72be63c7d1afc3
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:46.160671 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.137997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:49119 (local address 127.22.63.126:40883)
0806 01:06:46.138507 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.138512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.139634 (+ 1122us) server_negotiation.cc:407] Connection header received
0806 01:06:46.141701 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.141707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.141790 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.141907 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.143622 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.144814 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.146638 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.147462 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.147607 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.152295 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.152307 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.152316 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.152371 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.156667 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.157534 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.157544 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.157551 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.157762 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.158386 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.158394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.158399 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.158884 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.159072 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.159834 (+ 762us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.160147 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":275,"threads_started":1}
I20260806 01:06:46.161365 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.138461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40883 (local address 127.22.63.66:49119)
0806 01:06:46.139439 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:46.139488 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:46.141425 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:46.142186 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:46.142195 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:46.142774 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:46.143430 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:46.143449 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.144996 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.145000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:46.146490 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:46.146503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.147778 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.147790 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:46.148048 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.149001 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:46.149039 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:46.152023 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:46.156858 (+ 4835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:46.156867 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:46.156903 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:46.157367 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:46.157923 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:46.157927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:46.157929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:46.158240 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:46.159048 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:46.159055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:46.159396 (+ 341us) client_negotiation.cc:769] Sending connection context
0806 01:06:46.159642 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 01:06:46.160168 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":416,"threads_started":1}
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
I20260806 01:06:46.162927 23334 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40883
I20260806 01:06:46.163290 23334 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:46.164199 23334 heartbeater.cc:511] Master 127.22.63.126:40883 requested a full tablet report, sending...
I20260806 01:06:46.165822 22864 ts_manager.cc:194] Registered new tserver with Master: 22e876f906bc4cb69eea9f8e28833860 (127.22.63.66:37757)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
I20260806 01:06:46.166714 22864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:49119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:06:46.194995 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:06:46.532764 23129 heartbeater.cc:503] Master 127.22.63.126:40883 was elected leader, sending a full tablet report...
W20260806 01:06:46.554157 23340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:46.554441 23340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:46.554545 23340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:46.561949 23340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:46.562075 23340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:46.562263 23340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:06:46.572650 23340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40883
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:46.574007 23340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:46.575342 23340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:46.589192 23346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:46.589160 23347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:46.589162 23350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:46.590529 23340 server_base.cc:1034] running on GCE node
I20260806 01:06:46.591177 23340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:46.592293 23340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:46.593555 23340 hybrid_clock.cc:648] HybridClock initialized: now 1785978406593506 us; error 40 us; skew 500 ppm
I20260806 01:06:46.593894 23340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:46.597360 23340 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:06:46.599009 23340 webserver.cc:460] Webserver started at http://127.22.63.67:37123/ using document root <none> and password file <none>
I20260806 01:06:46.599804 23340 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:46.599977 23340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:46.600317 23340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:46.603343 23340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "cc1f035647a0410188c2d5deba8fd4ee"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "342e6ca830e1688122915774307ae877"
server_key_iv: "da172bf8713f81b0e89faeed92ab5397"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.604212 23340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "cc1f035647a0410188c2d5deba8fd4ee"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "342e6ca830e1688122915774307ae877"
server_key_iv: "da172bf8713f81b0e89faeed92ab5397"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.609939 23340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:06:46.613706 23361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.615319 23340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:06:46.615528 23340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "cc1f035647a0410188c2d5deba8fd4ee"
format_stamp: "Formatted at 2026-08-06 01:06:46 on dist-test-slave-7f01"
server_key: "342e6ca830e1688122915774307ae877"
server_key_iv: "da172bf8713f81b0e89faeed92ab5397"
server_key_version: "encryptionkey@0"
I20260806 01:06:46.615784 23340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:46.633083 23340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:46.637796 23340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:46.638139 23340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:46.639230 23340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:46.640779 23340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:46.640893 23340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.641060 23340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:46.641144 23340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:46.675786 23340 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:46809
I20260806 01:06:46.675835 23526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:46809 every 8 connection(s)
I20260806 01:06:46.677203 23340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:06:46.678072 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23340
I20260806 01:06:46.678309 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:06:46.678743 22781 external_mini_cluster.cc:1428] Setting key 1e0446821acb42ab08bb7d5e1a50c25d
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:46.699244 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.681575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:38281 (local address 127.22.63.126:40883)
0806 01:06:46.684150 (+ 2575us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.684154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.684172 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:06:46.684754 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.684759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.684831 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.684928 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.686485 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.687346 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.688948 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.689371 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.689468 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.693047 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.693053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.693057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.693098 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.696370 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.697151 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.697156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.697158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.697284 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.697683 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.697686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.697688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.697967 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.698090 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.698781 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.698956 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2461,"thread_start_us":2337,"threads_started":1}
I20260806 01:06:46.700350 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.680937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40883 (local address 127.22.63.67:38281)
0806 01:06:46.681904 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:46.681950 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:46.684517 (+ 2567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:46.685139 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:46.685147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:46.685730 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:46.686320 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:46.686337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.687509 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.687512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:46.688815 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:46.688827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.689597 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.689602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:46.689804 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.690514 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:46.690537 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:46.692787 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:46.696514 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:46.696521 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:46.696533 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:46.697006 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:46.697390 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:46.697393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:46.697396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:46.697580 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:46.698106 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:46.698113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:46.698441 (+ 328us) client_negotiation.cc:769] Sending connection context
0806 01:06:46.698675 (+ 234us) client_negotiation.cc:240] Negotiation successful
0806 01:06:46.699104 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":627,"threads_started":1}
I20260806 01:06:46.702149 23527 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40883
I20260806 01:06:46.702430 23527 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:46.703120 23527 heartbeater.cc:511] Master 127.22.63.126:40883 requested a full tablet report, sending...
I20260806 01:06:46.704631 22864 ts_manager.cc:194] Registered new tserver with Master: cc1f035647a0410188c2d5deba8fd4ee (127.22.63.67:46809)
I20260806 01:06:46.705500 22864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:38281
I20260806 01:06:46.708055 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:46.760041 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.743598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.22.63.65:38479)
0806 01:06:46.744644 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.744661 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.744690 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:06:46.744760 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.744767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.744984 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.745133 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.746584 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.747710 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.749162 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.749800 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.749917 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.754129 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.754141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.754157 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.754193 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.757200 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.757918 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.757926 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.757928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.758060 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.758427 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.758430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.758433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.758772 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.759009 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.759359 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.759599 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":532,"threads_started":1}
W20260806 01:06:46.761354 23004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33450
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:46.814700 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.798434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.22.63.65:38479)
0806 01:06:46.798725 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.798732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.798989 (+ 257us) server_negotiation.cc:407] Connection header received
0806 01:06:46.799174 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.799178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.799286 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.799385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.800831 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.801690 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.803560 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.804079 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.804184 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.809270 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.809275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.809279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.809319 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.812170 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.812904 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.812908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.812910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.813048 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.813411 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.813414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.813416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.813675 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.813883 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.814161 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.814346 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260806 01:06:46.816346 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22939
I20260806 01:06:46.829523 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.817310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.22.63.65:38479)
0806 01:06:46.817545 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.817550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.817781 (+ 231us) server_negotiation.cc:407] Connection header received
0806 01:06:46.817916 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.817920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.817976 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.818053 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.819182 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.819963 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.821333 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.821880 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.822010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.824653 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.824657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.824661 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.824691 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.827108 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.827801 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.827806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.827810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.827955 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.828281 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.828284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.828286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.828543 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.828720 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.828976 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.829131 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:06:46.830449 23124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:46.919296 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22939
I20260806 01:06:46.945652 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23139
Aug 06 01:06:46 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:06:46.963186 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:46.946798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.22.63.66:37757)
0806 01:06:46.947429 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:46.947436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:46.947464 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:06:46.947534 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:46.947540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:46.947798 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:46.947944 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:46.949306 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.950536 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.951897 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:46.952515 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:46.952623 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:46.957192 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:46.957207 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:46.957224 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:46.957265 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:46.960383 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.961062 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.961070 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.961072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.961210 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:46.961660 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:46.961663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:46.961665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:46.962020 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:46.962245 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:46.962632 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 01:06:46.962844 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":343,"threads_started":1}
I20260806 01:06:46.964767 23320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:47.051891 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23139
I20260806 01:06:47.075982 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23340
Aug 06 01:06:47 dist-test-slave-7f01 krb5kdc[22789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:06:47.093158 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:47.077056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42456 (local address 127.22.63.67:46809)
0806 01:06:47.077732 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:47.077739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:47.077785 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:06:47.077878 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:47.077885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:47.078111 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:47.078256 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:47.079902 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:47.080964 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:47.082230 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:47.082790 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:47.082893 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:47.087103 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:47.087117 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:47.087133 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:47.087170 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:47.090179 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:47.090951 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:47.090959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:47.090961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:47.091097 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:47.091503 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:47.091506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:47.091508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:47.091979 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:47.092229 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:47.092514 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 01:06:47.092719 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":354,"threads_started":1}
I20260806 01:06:47.094619 23522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:47.177623 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23340
I20260806 01:06:47.202205 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22813
I20260806 01:06:47.203478 22900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:47.309221 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22813
2026-08-06T01:06:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4563 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:47 dist-test-slave-7f01 krb5kdc[23558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:47 dist-test-slave-7f01 krb5kdc[23558](info): set up 2 sockets
Aug 06 01:06:47 dist-test-slave-7f01 krb5kdc[23558](info): commencing operation
krb5kdc: starting...
W20260806 01:06:48.614836 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:06:48 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:06:48.839671 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42835
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:42835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:49.177511 23573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:49.177907 23573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:49.178072 23573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:49.185403 23573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:49.185568 23573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:49.185685 23573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:49.185797 23573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:49.185894 23573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:49.196125 23573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:42835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:49.197592 23573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:49.199016 23573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:49.211449 23580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:49.211876 23581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:49.211884 23583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:49.212873 23573 server_base.cc:1034] running on GCE node
I20260806 01:06:49.213761 23573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:49.215247 23573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:49.216488 23573 hybrid_clock.cc:648] HybridClock initialized: now 1785978409216445 us; error 61 us; skew 500 ppm
I20260806 01:06:49.216866 23573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:49 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978409, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:49.220184 23573 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:06:49.221858 23573 webserver.cc:460] Webserver started at http://127.22.63.126:34035/ using document root <none> and password file <none>
I20260806 01:06:49.222589 23573 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:49.222744 23573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:49.223104 23573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:49.226295 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "284b07c9a31a40aa9dfc677cb59f623f"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "2c2c02cf4c4a22d6b87ac76b9c7bb71e"
server_key_iv: "97f7fd303a654b3bf8f9e19029959961"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.227108 23573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "284b07c9a31a40aa9dfc677cb59f623f"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "2c2c02cf4c4a22d6b87ac76b9c7bb71e"
server_key_iv: "97f7fd303a654b3bf8f9e19029959961"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.232528 23573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:06:49.236232 23591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:49.237600 23573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:06:49.237805 23573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "284b07c9a31a40aa9dfc677cb59f623f"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "2c2c02cf4c4a22d6b87ac76b9c7bb71e"
server_key_iv: "97f7fd303a654b3bf8f9e19029959961"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.238024 23573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:49.268292 23573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:49.272681 23573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:49.273013 23573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:49.292980 23573 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:42835
I20260806 01:06:49.293017 23675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:42835 every 8 connection(s)
I20260806 01:06:49.294466 23573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:49.298288 23676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:49.302182 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23573
I20260806 01:06:49.302430 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:06:49.303292 22781 external_mini_cluster.cc:1428] Setting key 060628e5666008fc9250ed41b6519d34
I20260806 01:06:49.312089 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f: Bootstrap starting.
I20260806 01:06:49.315411 23676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:49.316541 23676 log.cc:824] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:06:49 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:49.320127 23676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f: No bootstrap required, opened a new log
I20260806 01:06:49.327906 23682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:49.305584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.22.63.126:42835)
0806 01:06:49.306363 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:49.306375 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:49.306415 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:06:49.308151 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:49.308175 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:49.308576 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:49.308985 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:49.310523 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.311970 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.313266 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.313976 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.314170 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:49.318578 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:49.318591 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:49.318613 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:49.318658 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:49.322770 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:49.323547 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:49.323558 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:49.323572 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:49.323804 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:49.324236 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:49.324241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:49.324245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:49.324998 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:49.325281 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:49.325804 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 01:06:49.326347 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":351,"threads_started":1}
I20260806 01:06:49.330730 23676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } }
I20260806 01:06:49.331214 23676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:49.331346 23676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284b07c9a31a40aa9dfc677cb59f623f, State: Initialized, Role: FOLLOWER
I20260806 01:06:49.331951 23676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } }
I20260806 01:06:49.332209 23676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:49.332355 23676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:49.332527 23676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:49.334882 23676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } }
I20260806 01:06:49.335372 23676 leader_election.cc:302] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 284b07c9a31a40aa9dfc677cb59f623f; no voters:
I20260806 01:06:49.336155 23676 leader_election.cc:288] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:49.336337 23687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:49.338150 23687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [term 1 LEADER]: Becoming Leader. State: Replica: 284b07c9a31a40aa9dfc677cb59f623f, State: Running, Role: LEADER
I20260806 01:06:49.338842 23687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } }
I20260806 01:06:49.340724 23676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:49.342077 23688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 284b07c9a31a40aa9dfc677cb59f623f. Latest consensus state: current_term: 1 leader_uuid: "284b07c9a31a40aa9dfc677cb59f623f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } } }
I20260806 01:06:49.342545 23688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:49.344026 23690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "284b07c9a31a40aa9dfc677cb59f623f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "284b07c9a31a40aa9dfc677cb59f623f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42835 } } }
I20260806 01:06:49.344511 23690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:49.346719 23694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:49.351541 23694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:49.361261 23694 catalog_manager.cc:1347] Generated new cluster ID: 9406c21e3d8b4e55bf9ecd2325ccfb01
I20260806 01:06:49.361405 23694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:49.374913 23694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:49.376834 23694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:49.384852 23694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 284b07c9a31a40aa9dfc677cb59f623f: Generated new TSK 0
I20260806 01:06:49.385932 23694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:06:49.444401 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:49.799671 23710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:49.800051 23710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:49.800221 23710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:49.807848 23710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:49.807953 23710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:49.808118 23710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:06:49.818111 23710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:49.819525 23710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:49.820978 23710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:49.835417 23721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:49.835417 23722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:49.835475 23724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:49.836843 23710 server_base.cc:1034] running on GCE node
I20260806 01:06:49.837553 23710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:49.838585 23710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:49.839890 23710 hybrid_clock.cc:648] HybridClock initialized: now 1785978409839812 us; error 35 us; skew 500 ppm
I20260806 01:06:49.840278 23710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:49 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978409, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:49.844437 23710 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:06:49.846477 23710 webserver.cc:460] Webserver started at http://127.22.63.65:36681/ using document root <none> and password file <none>
I20260806 01:06:49.847276 23710 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:49.847435 23710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:49.847848 23710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:49.851042 23710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "978115d9fc4047e7986054844bb57104"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "bd6a95cb732e2f6e076937be3491dc72"
server_key_iv: "ebe3b057226d6258ed49e3b9e22a4d26"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.851999 23710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "978115d9fc4047e7986054844bb57104"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "bd6a95cb732e2f6e076937be3491dc72"
server_key_iv: "ebe3b057226d6258ed49e3b9e22a4d26"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.857653 23710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:06:49.861445 23732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:49.862854 23710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:06:49.863051 23710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "978115d9fc4047e7986054844bb57104"
format_stamp: "Formatted at 2026-08-06 01:06:49 on dist-test-slave-7f01"
server_key: "bd6a95cb732e2f6e076937be3491dc72"
server_key_iv: "ebe3b057226d6258ed49e3b9e22a4d26"
server_key_version: "encryptionkey@0"
I20260806 01:06:49.863269 23710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:49.880198 23710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:49.886143 23710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:49.886492 23710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:49.887611 23710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:49.889075 23710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:49.889189 23710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:49.889343 23710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:49.889423 23710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:49.923911 23710 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39115
I20260806 01:06:49.924103 23897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39115 every 8 connection(s)
I20260806 01:06:49.925315 23710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:06:49.927254 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23710
I20260806 01:06:49.927487 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:06:49.928341 22781 external_mini_cluster.cc:1428] Setting key 9740bfe1590405442d431d941ebbf658
Aug 06 01:06:49 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978409, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:49.947371 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:49.928173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:48651 (local address 127.22.63.126:42835)
0806 01:06:49.928716 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:49.928722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:49.929800 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 01:06:49.931819 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:49.931824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:49.931911 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:49.932055 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:49.933663 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.934403 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.936098 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.936533 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.936680 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:49.940742 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:49.940747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:49.940750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:49.940810 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:49.943926 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:49.944760 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:49.944767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:49.944771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:49.944920 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:49.945398 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:49.945404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:49.945408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:49.945764 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:49.945923 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:49.946725 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 01:06:49.947002 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":264,"threads_started":1}
I20260806 01:06:49.948547 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:49.928740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42835 (local address 127.22.63.65:48651)
0806 01:06:49.929644 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:49.929686 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:49.931520 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:49.932307 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:49.932316 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:49.932859 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:49.933486 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:49.933503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.934600 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.934604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:49.935946 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:49.935966 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:49.936783 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:49.936792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:49.937048 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:49.937803 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:49.937828 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:49.940440 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:49.944112 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:49.944124 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:49.944150 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:49.944628 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:49.945049 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:49.945054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:49.945057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:49.945278 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:49.945900 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:49.945909 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:49.946306 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 01:06:49.946612 (+ 306us) client_negotiation.cc:240] Negotiation successful
0806 01:06:49.947276 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":334,"threads_started":1}
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
I20260806 01:06:49.950328 23899 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42835
I20260806 01:06:49.950675 23899 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:49.951486 23899 heartbeater.cc:511] Master 127.22.63.126:42835 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
I20260806 01:06:49.953815 23616 ts_manager.cc:194] Registered new tserver with Master: 978115d9fc4047e7986054844bb57104 (127.22.63.65:39115)
I20260806 01:06:49.955283 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:48651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:06:49.983533 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:50.350346 23910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:50.350642 23910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:50.350924 23910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:50.358428 23910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:50.358548 23910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:50.358747 23910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:06:50.368990 23910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:50.370388 23910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:50.371836 23910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:50.385586 23922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:50.385862 23916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:50.385867 23920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:50.386610 23910 server_base.cc:1034] running on GCE node
I20260806 01:06:50.387352 23910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:50.388428 23910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:50.389685 23910 hybrid_clock.cc:648] HybridClock initialized: now 1785978410389632 us; error 45 us; skew 500 ppm
I20260806 01:06:50.390048 23910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:50 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978410, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:50.393389 23910 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:06:50.394894 23910 webserver.cc:460] Webserver started at http://127.22.63.66:34087/ using document root <none> and password file <none>
I20260806 01:06:50.395625 23910 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:50.395803 23910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:50.396220 23910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:50.399264 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "176702c1f5ee4780afca99b01044e1ec"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "ac05bb8c43dda1506582fc0423506f55"
server_key_iv: "76bb81fb342635e76284921df66f46bc"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.400148 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "176702c1f5ee4780afca99b01044e1ec"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "ac05bb8c43dda1506582fc0423506f55"
server_key_iv: "76bb81fb342635e76284921df66f46bc"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.405387 23910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:06:50.409112 23931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:50.410524 23910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:50.410717 23910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "176702c1f5ee4780afca99b01044e1ec"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "ac05bb8c43dda1506582fc0423506f55"
server_key_iv: "76bb81fb342635e76284921df66f46bc"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.410941 23910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:50.434686 23910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:50.439381 23910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:50.439669 23910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:50.440820 23910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:50.442655 23910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:50.442735 23910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:50.442836 23910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:50.442881 23910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:50.476456 23910 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39837
I20260806 01:06:50.476560 24091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39837 every 8 connection(s)
I20260806 01:06:50.477826 23910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:06:50.488230 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23910
I20260806 01:06:50.488441 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:06:50.488965 22781 external_mini_cluster.cc:1428] Setting key 862f91a669f78b7a4fa8d62e097a457f
Aug 06 01:06:50 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978410, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:50.499604 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:50.480558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:60379 (local address 127.22.63.126:42835)
0806 01:06:50.481047 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:50.481052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:50.481892 (+ 840us) server_negotiation.cc:407] Connection header received
0806 01:06:50.483845 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:50.483851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:50.483955 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:50.484090 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:50.485698 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:50.486443 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:50.488207 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:50.488797 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:50.488909 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:50.493001 (+ 4092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:50.493006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:50.493011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:50.493046 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:50.495960 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:50.497069 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:50.497074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:50.497076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:50.497222 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:50.497724 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:50.497728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:50.497730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:50.498030 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:50.498199 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:50.498946 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 01:06:50.499190 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":265,"threads_started":1}
I20260806 01:06:50.500635 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:50.480992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42835 (local address 127.22.63.66:60379)
0806 01:06:50.481731 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:50.481769 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:50.483536 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:50.484347 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:50.484355 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:50.484908 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:50.485515 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:50.485533 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:50.486606 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:50.486616 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:50.488045 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:50.488064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:50.489061 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:50.489070 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:50.489387 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:50.490195 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:50.490234 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:50.492742 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:50.496128 (+ 3386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:50.496138 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:50.496158 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:50.496892 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:50.497352 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:50.497358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:50.497362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:50.497580 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:50.498172 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:50.498182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:50.498564 (+ 382us) client_negotiation.cc:769] Sending connection context
0806 01:06:50.498846 (+ 282us) client_negotiation.cc:240] Negotiation successful
0806 01:06:50.499437 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":279,"threads_started":1}
I20260806 01:06:50.502195 24092 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42835
I20260806 01:06:50.502554 24092 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:50.503518 24092 heartbeater.cc:511] Master 127.22.63.126:42835 requested a full tablet report, sending...
I20260806 01:06:50.505406 23616 ts_manager.cc:194] Registered new tserver with Master: 176702c1f5ee4780afca99b01044e1ec (127.22.63.66:39837)
I20260806 01:06:50.506263 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:60379
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:06:50.541448 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:50.885407 24098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:50.885702 24098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:50.885804 24098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:50.893653 24098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:50.893766 24098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:50.893882 24098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:06:50.904549 24098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42835
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:50.905773 24098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:50.907099 24098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:50.920868 24108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:50.920935 24105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:50.921002 24106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:50.922117 24098 server_base.cc:1034] running on GCE node
I20260806 01:06:50.922731 24098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:50.923800 24098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:50.925067 24098 hybrid_clock.cc:648] HybridClock initialized: now 1785978410924998 us; error 58 us; skew 500 ppm
I20260806 01:06:50.925424 24098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:50 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978410, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:50.929189 24098 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:06:50.930624 24098 webserver.cc:460] Webserver started at http://127.22.63.67:33505/ using document root <none> and password file <none>
I20260806 01:06:50.931284 24098 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:50.931387 24098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:50.931653 24098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:50.934554 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "3c8c5ceec1c24fafa398e9f6b129916b"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "0ab07b074243a490b870096e020ede43"
server_key_iv: "c4ca15f6325a72d471360d6585298a82"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.935307 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "3c8c5ceec1c24fafa398e9f6b129916b"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "0ab07b074243a490b870096e020ede43"
server_key_iv: "c4ca15f6325a72d471360d6585298a82"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.940455 24098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:06:50.944697 24116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:50.946209 24098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:50.946393 24098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "3c8c5ceec1c24fafa398e9f6b129916b"
format_stamp: "Formatted at 2026-08-06 01:06:50 on dist-test-slave-7f01"
server_key: "0ab07b074243a490b870096e020ede43"
server_key_iv: "c4ca15f6325a72d471360d6585298a82"
server_key_version: "encryptionkey@0"
I20260806 01:06:50.946617 24098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:50.958436 23899 heartbeater.cc:503] Master 127.22.63.126:42835 was elected leader, sending a full tablet report...
I20260806 01:06:50.963207 24098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:50.967370 24098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:50.967677 24098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:50.968828 24098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:06:50.970160 24098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:50.970233 24098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:50.970320 24098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:50.970366 24098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:51.007035 24098 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:33185
I20260806 01:06:51.007130 24280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:33185 every 8 connection(s)
I20260806 01:06:51.008476 24098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:06:51.014066 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24098
I20260806 01:06:51.014272 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:06:51.014853 22781 external_mini_cluster.cc:1428] Setting key 209a512d68698eba925a23442824f469
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978410, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:51.029775 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.011236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:41391 (local address 127.22.63.126:42835)
0806 01:06:51.011679 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.011684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.012765 (+ 1081us) server_negotiation.cc:407] Connection header received
0806 01:06:51.015135 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.015141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.015284 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.015402 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.017378 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.018289 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.019852 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.020267 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.020365 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.023815 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.023819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.023823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.023862 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.026582 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.027279 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.027283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.027285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.027406 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.027923 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.027926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.027928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.028192 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.028311 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.029140 (+ 829us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.029381 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
I20260806 01:06:51.030903 24289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.011883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42835 (local address 127.22.63.67:41391)
0806 01:06:51.012597 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.012640 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.014764 (+ 2124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.015657 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.015666 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.016416 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:51.017172 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.017214 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.018430 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.018434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.019655 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.019667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.020457 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.020462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.020667 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.021372 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:51.021395 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:51.023568 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:51.026710 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:51.026717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:51.026728 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:51.027170 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:51.027532 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:51.027536 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:51.027538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:51.027734 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:51.028310 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:51.028316 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:51.028704 (+ 388us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.029044 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.029548 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":265,"threads_started":1}
I20260806 01:06:51.032287 24281 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42835
I20260806 01:06:51.032560 24281 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:51.033231 24281 heartbeater.cc:511] Master 127.22.63.126:42835 requested a full tablet report, sending...
I20260806 01:06:51.034657 23616 ts_manager.cc:194] Registered new tserver with Master: 3c8c5ceec1c24fafa398e9f6b129916b (127.22.63.67:33185)
I20260806 01:06:51.035411 23616 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:41391
I20260806 01:06:51.042913 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:06:51.063890 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.050739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.22.63.126:42835)
0806 01:06:51.050921 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.050926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.051559 (+ 633us) server_negotiation.cc:407] Connection header received
0806 01:06:51.051730 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.051774 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.051867 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.051951 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.053459 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.054205 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.055484 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.055961 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.056059 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.058876 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.058880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.058884 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.058918 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.061523 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.062107 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.062111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.062113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.062233 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.062559 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.062562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.062564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.062866 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.063041 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.063357 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.063531 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:06:51.071707 23616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:06:51.078694 23616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:06:51.120113 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.091966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.22.63.66:39837)
0806 01:06:51.092949 (+ 983us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.092961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.103509 (+ 10548us) server_negotiation.cc:407] Connection header received
0806 01:06:51.105600 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.105613 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.105968 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.106190 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:51.108700 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.111040 (+ 2340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.115287 (+ 4247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.117397 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.117552 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.118064 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.119526 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.119725 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":777,"thread_start_us":1501,"threads_started":1}
I20260806 01:06:51.120563 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.092001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39837 (local address 127.0.0.1:57178)
0806 01:06:51.103259 (+ 11258us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.103332 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.104953 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.106620 (+ 1667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.106632 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.106674 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:51.107397 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.107416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.111620 (+ 4204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.111629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.115052 (+ 3423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.115071 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.118378 (+ 3307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.118388 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.118712 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.118788 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.119825 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.120028 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10867,"mutex_wait_us":1,"thread_start_us":27377,"threads_started":1}
I20260806 01:06:51.122956 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.092379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33185 (local address 127.0.0.1:42614)
0806 01:06:51.110851 (+ 18472us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.110877 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.111051 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.112148 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.112152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.112179 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:51.112800 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.112818 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.115659 (+ 2841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.115664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.119255 (+ 3591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.119272 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.122113 (+ 2841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.122120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.122305 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.122339 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.122433 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.122571 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18296,"mutex_wait_us":438,"thread_start_us":10289,"threads_started":1}
I20260806 01:06:51.123172 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.093162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.22.63.67:33185)
0806 01:06:51.109887 (+ 16725us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.109901 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.111187 (+ 1286us) server_negotiation.cc:407] Connection header received
0806 01:06:51.111310 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.111318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.111587 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.111834 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:51.113142 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.115481 (+ 2339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.119465 (+ 3984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.121786 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.121915 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.122151 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.122550 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.122716 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16406,"thread_start_us":16788,"threads_started":1}
I20260806 01:06:51.125033 24000 tablet_service.cc:1467] Processing CreateTablet for tablet 67d9d34d3c394d8da4718b5488692c7c (DEFAULT_TABLE table=test-table [id=69d370079b10432f83ef8d2550f1cd61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:51.127306 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.092654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:39115 (local address 127.0.0.1:35982)
0806 01:06:51.119778 (+ 27124us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.119798 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.119927 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.120966 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.120971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.121011 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:51.121354 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.121366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.123323 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.123328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.124954 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.124966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.126368 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.126375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.126544 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.126594 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.126721 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.126873 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26920,"thread_start_us":18549,"threads_started":1}
I20260806 01:06:51.127358 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.093221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.22.63.65:39115)
0806 01:06:51.119167 (+ 25946us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.119174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.120077 (+ 903us) server_negotiation.cc:407] Connection header received
0806 01:06:51.120186 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.120194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.120498 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.120714 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:51.121549 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.123137 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.125132 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.126143 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.126303 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.126593 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.126775 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.126939 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25769,"thread_start_us":25608,"threads_started":1}
I20260806 01:06:51.126973 24185 tablet_service.cc:1467] Processing CreateTablet for tablet 67d9d34d3c394d8da4718b5488692c7c (DEFAULT_TABLE table=test-table [id=69d370079b10432f83ef8d2550f1cd61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:51.128034 24000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d9d34d3c394d8da4718b5488692c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:51.128590 24185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d9d34d3c394d8da4718b5488692c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:51.130724 23811 tablet_service.cc:1467] Processing CreateTablet for tablet 67d9d34d3c394d8da4718b5488692c7c (DEFAULT_TABLE table=test-table [id=69d370079b10432f83ef8d2550f1cd61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:51.131982 23811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67d9d34d3c394d8da4718b5488692c7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:51.148576 24313 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Bootstrap starting.
I20260806 01:06:51.149498 24315 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Bootstrap starting.
I20260806 01:06:51.150388 24317 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Bootstrap starting.
I20260806 01:06:51.155216 24317 tablet_bootstrap.cc:654] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:51.156208 24313 tablet_bootstrap.cc:654] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:51.156876 24315 tablet_bootstrap.cc:654] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:51.158190 24313 log.cc:824] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:51.158306 24317 log.cc:824] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:51.159658 24315 log.cc:824] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:51.162668 24313 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: No bootstrap required, opened a new log
I20260806 01:06:51.163159 24313 ts_tablet_manager.cc:1397] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260806 01:06:51.164112 24317 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: No bootstrap required, opened a new log
I20260806 01:06:51.164306 24315 tablet_bootstrap.cc:492] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: No bootstrap required, opened a new log
I20260806 01:06:51.164718 24315 ts_tablet_manager.cc:1397] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.001s
I20260806 01:06:51.164737 24317 ts_tablet_manager.cc:1397] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260806 01:06:51.176712 24315 raft_consensus.cc:348] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.177335 24315 raft_consensus.cc:374] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:51.177628 24315 raft_consensus.cc:729] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176702c1f5ee4780afca99b01044e1ec, State: Initialized, Role: FOLLOWER
I20260806 01:06:51.178591 24315 consensus_queue.cc:260] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.180509 24092 heartbeater.cc:503] Master 127.22.63.126:42835 was elected leader, sending a full tablet report...
I20260806 01:06:51.181380 24315 ts_tablet_manager.cc:1428] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20260806 01:06:51.183048 24313 raft_consensus.cc:348] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.183140 24317 raft_consensus.cc:348] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.183653 24313 raft_consensus.cc:374] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:51.183822 24313 raft_consensus.cc:729] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978115d9fc4047e7986054844bb57104, State: Initialized, Role: FOLLOWER
I20260806 01:06:51.183845 24317 raft_consensus.cc:374] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:51.183955 24317 raft_consensus.cc:729] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8c5ceec1c24fafa398e9f6b129916b, State: Initialized, Role: FOLLOWER
I20260806 01:06:51.184731 24317 consensus_queue.cc:260] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.184993 24313 consensus_queue.cc:260] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.187558 24313 ts_tablet_manager.cc:1428] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Time spent starting tablet: real 0.024s user 0.008s sys 0.014s
I20260806 01:06:51.188730 24281 heartbeater.cc:503] Master 127.22.63.126:42835 was elected leader, sending a full tablet report...
I20260806 01:06:51.189193 24317 ts_tablet_manager.cc:1428] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Time spent starting tablet: real 0.024s user 0.013s sys 0.011s
I20260806 01:06:51.211102 24325 raft_consensus.cc:482] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:06:51.211519 24325 raft_consensus.cc:504] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.213357 24325 leader_election.cc:288] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176702c1f5ee4780afca99b01044e1ec (127.22.63.66:39837), 3c8c5ceec1c24fafa398e9f6b129916b (127.22.63.67:33185)
I20260806 01:06:51.223796 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.214058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:59161 (local address 127.22.63.66:39837)
0806 01:06:51.214330 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.214335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.215123 (+ 788us) server_negotiation.cc:407] Connection header received
0806 01:06:51.215470 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.215475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.215574 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.215685 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:51.216602 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.218035 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.221222 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.222523 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.222680 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.222754 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.223352 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.223488 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:06:51.223807 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.213974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:48201 (local address 127.22.63.67:33185)
0806 01:06:51.214298 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.214307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.215123 (+ 816us) server_negotiation.cc:407] Connection header received
0806 01:06:51.215508 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.215513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.215588 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.215684 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:51.216547 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.217884 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.221211 (+ 3327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.222496 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.222676 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.222754 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.223353 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.223488 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:06:51.224064 24328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.213670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39837 (local address 127.22.63.65:59161)
0806 01:06:51.214975 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.215000 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.215291 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.215719 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.215723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.215782 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:51.216446 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.216460 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.218154 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.218161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.221006 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.221023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.222840 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.222848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.223085 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.223200 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.223304 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.223500 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":318,"spinlock_wait_cycles":290816,"thread_start_us":888,"threads_started":1}
I20260806 01:06:51.224478 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.213745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33185 (local address 127.22.63.65:48201)
0806 01:06:51.214999 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:51.215014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:51.215294 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:51.215719 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:51.215724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:51.215795 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:51.216320 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.216337 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.218112 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.218120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.221006 (+ 2886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:51.221023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.222841 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.222848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:51.223085 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.223195 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 01:06:51.223308 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:06:51.223500 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"mutex_wait_us":261,"spinlock_wait_cycles":896,"thread_start_us":982,"threads_started":1}
I20260806 01:06:51.225351 24025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d9d34d3c394d8da4718b5488692c7c" candidate_uuid: "978115d9fc4047e7986054844bb57104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176702c1f5ee4780afca99b01044e1ec" is_pre_election: true
I20260806 01:06:51.225653 24217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d9d34d3c394d8da4718b5488692c7c" candidate_uuid: "978115d9fc4047e7986054844bb57104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" is_pre_election: true
I20260806 01:06:51.226003 24025 raft_consensus.cc:2454] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978115d9fc4047e7986054844bb57104 in term 0.
I20260806 01:06:51.226219 24217 raft_consensus.cc:2454] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 978115d9fc4047e7986054844bb57104 in term 0.
I20260806 01:06:51.226812 23736 leader_election.cc:302] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176702c1f5ee4780afca99b01044e1ec, 978115d9fc4047e7986054844bb57104; no voters:
I20260806 01:06:51.227202 24325 raft_consensus.cc:2781] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:06:51.227360 24325 raft_consensus.cc:482] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:06:51.227470 24325 raft_consensus.cc:3037] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:51.229807 24325 raft_consensus.cc:504] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
W20260806 01:06:51.229995 24093 tablet.cc:2369] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:51.230563 24325 leader_election.cc:288] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [CANDIDATE]: Term 1 election: Requested vote from peers 176702c1f5ee4780afca99b01044e1ec (127.22.63.66:39837), 3c8c5ceec1c24fafa398e9f6b129916b (127.22.63.67:33185)
I20260806 01:06:51.231019 24025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d9d34d3c394d8da4718b5488692c7c" candidate_uuid: "978115d9fc4047e7986054844bb57104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176702c1f5ee4780afca99b01044e1ec"
I20260806 01:06:51.231242 24025 raft_consensus.cc:3037] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:51.231431 24217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67d9d34d3c394d8da4718b5488692c7c" candidate_uuid: "978115d9fc4047e7986054844bb57104" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c8c5ceec1c24fafa398e9f6b129916b"
I20260806 01:06:51.231688 24217 raft_consensus.cc:3037] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:51.234061 24025 raft_consensus.cc:2454] T 67d9d34d3c394d8da4718b5488692c7c P 176702c1f5ee4780afca99b01044e1ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978115d9fc4047e7986054844bb57104 in term 1.
I20260806 01:06:51.234360 24217 raft_consensus.cc:2454] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 978115d9fc4047e7986054844bb57104 in term 1.
I20260806 01:06:51.234589 23736 leader_election.cc:302] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176702c1f5ee4780afca99b01044e1ec, 978115d9fc4047e7986054844bb57104; no voters:
I20260806 01:06:51.234928 24325 raft_consensus.cc:2781] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:51.235426 24325 raft_consensus.cc:686] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [term 1 LEADER]: Becoming Leader. State: Replica: 978115d9fc4047e7986054844bb57104, State: Running, Role: LEADER
I20260806 01:06:51.236003 24325 consensus_queue.cc:237] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } }
I20260806 01:06:51.241214 23614 catalog_manager.cc:5495] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104 reported cstate change: term changed from 0 to 1, leader changed from <none> to 978115d9fc4047e7986054844bb57104 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "978115d9fc4047e7986054844bb57104" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176702c1f5ee4780afca99b01044e1ec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c8c5ceec1c24fafa398e9f6b129916b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "978115d9fc4047e7986054844bb57104" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39115 } health_report { overall_health: HEALTHY } } }
W20260806 01:06:51.260478 24283 tablet.cc:2369] T 67d9d34d3c394d8da4718b5488692c7c P 3c8c5ceec1c24fafa398e9f6b129916b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:51.281349 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.265478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.22.63.65:39115)
0806 01:06:51.265744 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.265753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.266134 (+ 381us) server_negotiation.cc:407] Connection header received
0806 01:06:51.266361 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.266365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.266456 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.266585 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.268309 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.269058 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.270358 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.270783 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.270925 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.275205 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.275220 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.275237 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.275274 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.278321 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.279065 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.279072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.279075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.279237 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.279639 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.279644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.279647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.280099 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.280422 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.280725 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.281025 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:51.338539 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.324125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.22.63.65:39115)
0806 01:06:51.324321 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.324325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.324802 (+ 477us) server_negotiation.cc:407] Connection header received
0806 01:06:51.324944 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.324948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.325037 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.325135 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.326500 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.327266 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.328619 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.329032 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.329128 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.333321 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.333326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.333330 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.333368 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.336169 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.336784 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.336788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.336790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.336958 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.337333 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.337336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.337338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.337596 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.337846 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.338083 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.338271 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:51.392016 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.377544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.22.63.65:39115)
0806 01:06:51.377827 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.377834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.378050 (+ 216us) server_negotiation.cc:407] Connection header received
0806 01:06:51.378205 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.378209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.378307 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.378396 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.379821 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.380601 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.382007 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.382421 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.382515 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.386619 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.386624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.386627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.386664 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.389389 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.390103 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.390107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.390109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.390231 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.390576 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.390579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.390581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.390851 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.391076 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.391388 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.391567 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260806 01:06:51.393906 23780 tablet_service.cc:2908] Rejecting scan request for tablet 67d9d34d3c394d8da4718b5488692c7c: Uninitialized: clean time has not yet been initialized
I20260806 01:06:51.398307 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23710
I20260806 01:06:51.411531 24311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.399220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.22.63.65:39115)
0806 01:06:51.399481 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.399491 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.399842 (+ 351us) server_negotiation.cc:407] Connection header received
0806 01:06:51.400013 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.400017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.400107 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.400196 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.401388 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.402097 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.403279 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.403667 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.403858 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.405980 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.405985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.405989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.406022 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.408871 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.409571 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.409576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.409578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.409716 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.410100 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.410108 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.410114 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.410505 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.410707 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.410959 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.411155 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:06:51.412230 23893 generic_service.cc:196] Checking for leaks (request via RPC)
W20260806 01:06:51.510497 23900 tablet.cc:2369] T 67d9d34d3c394d8da4718b5488692c7c P 978115d9fc4047e7986054844bb57104: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:51.511788 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23710
I20260806 01:06:51.538627 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23910
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:06:51.557843 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.539520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.22.63.66:39837)
0806 01:06:51.539730 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.539767 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.539791 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:06:51.539901 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.539905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.539983 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.540095 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.541496 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.542331 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.543820 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.544305 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.544457 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.550112 (+ 5655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.550134 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.550180 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.550231 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.554470 (+ 4239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.555462 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.555474 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.555480 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.555640 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.556106 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.556113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.556118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.556587 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.556961 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.557205 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.557444 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:06:51.558604 24085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:51.652365 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23910
I20260806 01:06:51.682085 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24098
Aug 06 01:06:51 dist-test-slave-7f01 krb5kdc[23558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:06:51.697577 24312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:51.682968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.22.63.67:33185)
0806 01:06:51.683165 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:51.683170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:51.683190 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:06:51.683318 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:51.683322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:51.683391 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:51.683495 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:51.684923 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.685674 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.687001 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:51.687383 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:51.687475 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:51.691679 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:51.691693 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:51.691709 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:51.691805 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:51.695037 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.695672 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.695679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.695681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.695920 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:51.696297 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:51.696302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:51.696304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:51.696667 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:51.696901 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:51.697150 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 01:06:51.697320 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:06:51.698310 24275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:51.786724 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24098
I20260806 01:06:51.813241 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23573
I20260806 01:06:51.814396 23670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:51.926712 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23573
2026-08-06T01:06:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4617 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:52 dist-test-slave-7f01 krb5kdc[24360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:52 dist-test-slave-7f01 krb5kdc[24360](info): set up 2 sockets
Aug 06 01:06:52 dist-test-slave-7f01 krb5kdc[24360](info): commencing operation
krb5kdc: starting...
W20260806 01:06:53.238101 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:06:53 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:06:53.469108 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40477
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:53.806049 24378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:53.806414 24378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:53.806593 24378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:53.813877 24378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:53.814024 24378 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:06:53.814117 24378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:53.814237 24378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:53.814328 24378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:53.814436 24378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:53.824554 24378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40477
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:53.825973 24378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:53.827433 24378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:53.839808 24383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:53.839725 24386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:53.839694 24384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:53.841051 24378 server_base.cc:1034] running on GCE node
I20260806 01:06:53.841987 24378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:53.843423 24378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:53.844653 24378 hybrid_clock.cc:648] HybridClock initialized: now 1785978413844572 us; error 55 us; skew 500 ppm
I20260806 01:06:53.844985 24378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:53 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978413, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:53.848443 24378 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:06:53.850175 24378 webserver.cc:460] Webserver started at http://127.22.63.126:41151/ using document root <none> and password file <none>
I20260806 01:06:53.850963 24378 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:53.851101 24378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:53.851475 24378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:53.854791 24378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "b7ef5acb21c54d0e86ea1c131f7985ff"
format_stamp: "Formatted at 2026-08-06 01:06:53 on dist-test-slave-7f01"
server_key: "566fed9dc5da07cb0885cb5e34ce1d3a"
server_key_iv: "a2ab1a6d8cc5db4d0b7c13bdeb74edd5"
server_key_version: "encryptionkey@0"
I20260806 01:06:53.855620 24378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "b7ef5acb21c54d0e86ea1c131f7985ff"
format_stamp: "Formatted at 2026-08-06 01:06:53 on dist-test-slave-7f01"
server_key: "566fed9dc5da07cb0885cb5e34ce1d3a"
server_key_iv: "a2ab1a6d8cc5db4d0b7c13bdeb74edd5"
server_key_version: "encryptionkey@0"
I20260806 01:06:53.861037 24378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:06:53.864552 24396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:53.865875 24378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:53.866065 24378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "b7ef5acb21c54d0e86ea1c131f7985ff"
format_stamp: "Formatted at 2026-08-06 01:06:53 on dist-test-slave-7f01"
server_key: "566fed9dc5da07cb0885cb5e34ce1d3a"
server_key_iv: "a2ab1a6d8cc5db4d0b7c13bdeb74edd5"
server_key_version: "encryptionkey@0"
I20260806 01:06:53.866292 24378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:53.884740 24378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:53.888981 24378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:53.889295 24378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:53.912019 24378 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40477
I20260806 01:06:53.912041 24488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40477 every 8 connection(s)
I20260806 01:06:53.913533 24378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:53.915673 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24378
I20260806 01:06:53.915947 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:06:53.916831 22781 external_mini_cluster.cc:1428] Setting key 7c45c7b7eff02de122afe1741ee43710
I20260806 01:06:53.918576 24490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:06:53 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:53.932675 24490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Bootstrap starting.
I20260806 01:06:53.937493 24490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:53.938807 24490 log.cc:824] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:53.941995 24490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: No bootstrap required, opened a new log
I20260806 01:06:53.942619 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:53.919235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.22.63.126:40477)
0806 01:06:53.920083 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:53.920098 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:53.920148 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:06:53.922161 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:53.922188 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:53.922710 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:53.923151 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:53.924588 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:53.926688 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:53.928078 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:53.929127 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:53.929343 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:53.933606 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:53.933616 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:53.933634 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:53.933683 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:53.938201 (+ 4518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:53.938959 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:53.938968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:53.938976 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:53.939189 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:53.939633 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:53.939637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:53.939643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:53.940290 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:53.940622 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:53.941073 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 01:06:53.941508 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":334,"threads_started":1}
I20260806 01:06:53.950464 24490 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } }
I20260806 01:06:53.951051 24490 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:53.951185 24490 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ef5acb21c54d0e86ea1c131f7985ff, State: Initialized, Role: FOLLOWER
I20260806 01:06:53.951828 24490 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } }
I20260806 01:06:53.952054 24490 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:53.952157 24490 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:53.952292 24490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:53.954438 24490 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } }
I20260806 01:06:53.955054 24490 leader_election.cc:302] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7ef5acb21c54d0e86ea1c131f7985ff; no voters:
I20260806 01:06:53.955781 24490 leader_election.cc:288] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:53.955946 24495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:53.956684 24495 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [term 1 LEADER]: Becoming Leader. State: Replica: b7ef5acb21c54d0e86ea1c131f7985ff, State: Running, Role: LEADER
I20260806 01:06:53.957202 24495 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } }
I20260806 01:06:53.958043 24490 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:53.960196 24497 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } } }
I20260806 01:06:53.960835 24497 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:53.960613 24498 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ef5acb21c54d0e86ea1c131f7985ff. Latest consensus state: current_term: 1 leader_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef5acb21c54d0e86ea1c131f7985ff" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40477 } } }
I20260806 01:06:53.962149 24498 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:53.962682 24505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:53.968062 24505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:53.977588 24505 catalog_manager.cc:1347] Generated new cluster ID: 7f655a5e4f7f421ea4e9a7bebb594b23
I20260806 01:06:53.977715 24505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:54.009399 24505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:54.010795 24505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:54.028180 24505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Generated new TSK 0
I20260806 01:06:54.029143 24505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:06:54.096459 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40477
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:54.562659 24527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:54.562990 24527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:54.563174 24527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:54.571301 24527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:54.571441 24527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:54.571635 24527 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:06:54.571770 24527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:06:54.582432 24527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:54.583997 24527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:54.585474 24527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:54.600394 24534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:54.600661 24540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:54.601104 24533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:54.601922 24527 server_base.cc:1034] running on GCE node
I20260806 01:06:54.602818 24527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:54.604256 24527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:54.605556 24527 hybrid_clock.cc:648] HybridClock initialized: now 1785978414605454 us; error 67 us; skew 500 ppm
I20260806 01:06:54.606009 24527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:54 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978414, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:54.610608 24527 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:06:54.612427 24527 webserver.cc:460] Webserver started at http://127.22.63.65:32813/ using document root <none> and password file <none>
I20260806 01:06:54.613194 24527 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:54.613338 24527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:54.613688 24527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:54.616943 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "59b7ad32f1d84391bd62ff76007e5566"
format_stamp: "Formatted at 2026-08-06 01:06:54 on dist-test-slave-7f01"
server_key: "31477b8e83a967f5404cd5938ec2aa38"
server_key_iv: "b3f6ae89a04a90395a0002a6304b7981"
server_key_version: "encryptionkey@0"
I20260806 01:06:54.617826 24527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "59b7ad32f1d84391bd62ff76007e5566"
format_stamp: "Formatted at 2026-08-06 01:06:54 on dist-test-slave-7f01"
server_key: "31477b8e83a967f5404cd5938ec2aa38"
server_key_iv: "b3f6ae89a04a90395a0002a6304b7981"
server_key_version: "encryptionkey@0"
I20260806 01:06:54.623867 24527 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 01:06:54.628006 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:54.629738 24527 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:06:54.630008 24527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "59b7ad32f1d84391bd62ff76007e5566"
format_stamp: "Formatted at 2026-08-06 01:06:54 on dist-test-slave-7f01"
server_key: "31477b8e83a967f5404cd5938ec2aa38"
server_key_iv: "b3f6ae89a04a90395a0002a6304b7981"
server_key_version: "encryptionkey@0"
I20260806 01:06:54.630321 24527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:54.663338 24527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:54.668622 24527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:54.668955 24527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:54.672658 24527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:54.672770 24527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:54.672969 24527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:54.673058 24527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:06:54 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978414, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:54.715699 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:54.673560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54469 (local address 127.22.63.126:40477)
0806 01:06:54.686057 (+ 12497us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:54.686066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:54.687525 (+ 1459us) server_negotiation.cc:407] Connection header received
0806 01:06:54.690964 (+ 3439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:54.690974 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:54.691093 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:54.693767 (+ 2674us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:54.693953 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.695358 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.698865 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.699727 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.699931 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:54.706491 (+ 6560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:54.706502 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:54.706510 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:54.706567 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:54.710902 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:54.712289 (+ 1387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:54.712297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:54.712304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:54.712487 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:54.713214 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:54.713219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:54.713227 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:54.713716 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:54.713899 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:54.715061 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0806 01:06:54.715324 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12281,"thread_start_us":12503,"threads_started":1}
I20260806 01:06:54.717342 24563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:54.674239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.65:54469)
0806 01:06:54.687283 (+ 13044us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:54.687362 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:54.690520 (+ 3158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:54.691615 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:54.691631 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:54.692569 (+ 938us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:54.693543 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:54.693572 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.695613 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.695626 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:54.698655 (+ 3029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:54.698678 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.700091 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.700102 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:54.700534 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:54.701772 (+ 1238us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:54.701818 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:54.706113 (+ 4295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:54.711093 (+ 4980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:54.711103 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:54.711125 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:54.712073 (+ 948us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:54.712646 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:54.712654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:54.712660 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:54.713013 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:54.713909 (+ 896us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:54.713923 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:54.714532 (+ 609us) client_negotiation.cc:769] Sending connection context
0806 01:06:54.714941 (+ 409us) client_negotiation.cc:240] Negotiation successful
0806 01:06:54.715538 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12717,"thread_start_us":12472,"threads_started":1}
I20260806 01:06:54.742691 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:54.726116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:56157 (local address 127.22.63.126:40477)
0806 01:06:54.726374 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:54.726383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:54.727015 (+ 632us) server_negotiation.cc:407] Connection header received
0806 01:06:54.727182 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:54.727191 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:54.727313 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:54.727449 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:54.728768 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.729940 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.731289 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.731729 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.731931 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:54.734862 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:54.734871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:54.734879 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:54.734963 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:54.739212 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:54.740041 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:54.740047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:54.740051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:54.740244 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:54.740801 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:54.740808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:54.740813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:54.741247 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:54.741457 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:54.742051 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 01:06:54.742289 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:06:54.742767 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:54.726116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.65:56157)
0806 01:06:54.726914 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:54.726931 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:54.727055 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:54.727485 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:54.727489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:54.728041 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:54.728590 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:54.728609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.730096 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.730101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:54.731141 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:54.731153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:54.732009 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:54.732018 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:54.732303 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:54.733158 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:54.733189 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:54.734603 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:54.739389 (+ 4786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:54.739393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:54.739396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:54.739889 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:54.740385 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:54.740392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:54.740397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:54.740617 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:54.741449 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:54.741456 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:54.741738 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 01:06:54.741934 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 01:06:54.742294 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":397,"threads_started":1}
I20260806 01:06:54.754282 24527 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:45361
I20260806 01:06:54.754312 24721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:45361 every 8 connection(s)
I20260806 01:06:54.755625 24527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:06:54.757925 24722 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40477
I20260806 01:06:54.758167 24722 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:54.758834 24722 heartbeater.cc:511] Master 127.22.63.126:40477 requested a full tablet report, sending...
I20260806 01:06:54.760977 24417 ts_manager.cc:194] Registered new tserver with Master: 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361)
I20260806 01:06:54.762411 24417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:54469
I20260806 01:06:54.764205 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24527
I20260806 01:06:54.764449 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:06:54.765295 22781 external_mini_cluster.cc:1428] Setting key 1b6d51a4a9834ddf6a66ffb9a4e88012
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:06:54.815459 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40477
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:55.156031 24726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:55.156337 24726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:55.156564 24726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:55.163969 24726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:55.164080 24726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:55.164247 24726 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:06:55.164369 24726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:06:55.174638 24726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:55.176072 24726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:55.177501 24726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:55.191556 24739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:55.191573 24744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:55.191771 24738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:55.192284 24726 server_base.cc:1034] running on GCE node
I20260806 01:06:55.193214 24726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:55.194326 24726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:55.195654 24726 hybrid_clock.cc:648] HybridClock initialized: now 1785978415195553 us; error 61 us; skew 500 ppm
I20260806 01:06:55.196039 24726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:55.199532 24726 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:06:55.201279 24726 webserver.cc:460] Webserver started at http://127.22.63.66:41539/ using document root <none> and password file <none>
I20260806 01:06:55.202062 24726 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:55.202158 24726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:55.202541 24726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:55.205729 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "8e662941504842acaee5e5c488170dec"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "91ef4847628c405c9631558e176f6ccd"
server_key_iv: "a7fcd1053ee45bd09057a8e52ba6155d"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.206593 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "8e662941504842acaee5e5c488170dec"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "91ef4847628c405c9631558e176f6ccd"
server_key_iv: "a7fcd1053ee45bd09057a8e52ba6155d"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.212309 24726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:06:55.216387 24754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:55.217878 24726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:55.218183 24726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "8e662941504842acaee5e5c488170dec"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "91ef4847628c405c9631558e176f6ccd"
server_key_iv: "a7fcd1053ee45bd09057a8e52ba6155d"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.218442 24726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:55.253309 24726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:55.257884 24726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:55.258183 24726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:55.261726 24726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:55.261842 24726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:55.262029 24726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:55.262115 24726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:55.291287 24769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.262444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.66:34109)
0806 01:06:55.263433 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.263494 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.266330 (+ 2836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.267369 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.267384 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.268410 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:55.269550 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.269581 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.271501 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.271509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.274526 (+ 3017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.274550 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.275667 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.275677 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.276095 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.277137 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:55.277165 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:55.281021 (+ 3856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:55.285341 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.285352 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.285377 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.286217 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.286753 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.286757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.286759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.286994 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.287796 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:55.287808 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:55.288363 (+ 555us) client_negotiation.cc:769] Sending connection context
0806 01:06:55.288792 (+ 429us) client_negotiation.cc:240] Negotiation successful
0806 01:06:55.289509 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":375,"threads_started":1}
I20260806 01:06:55.292559 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.261681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:34109 (local address 127.22.63.126:40477)
0806 01:06:55.262335 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.262345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.263647 (+ 1302us) server_negotiation.cc:407] Connection header received
0806 01:06:55.266715 (+ 3068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.266720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.266861 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.266990 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:55.269826 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.271233 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.274725 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.275328 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.275510 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.281338 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:55.281344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:55.281347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:55.281425 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:55.285152 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.286408 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.286418 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.286422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.286614 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.287139 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.287145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.287148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.287614 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:55.287857 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:55.291980 (+ 4123us) server_negotiation.cc:299] Negotiation successful
0806 01:06:55.292277 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":330,"threads_started":1}
I20260806 01:06:55.314544 24930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39411 every 8 connection(s)
I20260806 01:06:55.314579 24726 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39411
I20260806 01:06:55.316494 24726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:06:55.316984 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24726
I20260806 01:06:55.317212 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:06:55.317832 22781 external_mini_cluster.cc:1428] Setting key bbc5626d48a66a76bc1b7fa43d4546e7
I20260806 01:06:55.319957 24931 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40477
I20260806 01:06:55.320283 24931 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:55.321199 24931 heartbeater.cc:511] Master 127.22.63.126:40477 requested a full tablet report, sending...
I20260806 01:06:55.322297 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.298905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:38253 (local address 127.22.63.126:40477)
0806 01:06:55.299119 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.299126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.305031 (+ 5905us) server_negotiation.cc:407] Connection header received
0806 01:06:55.305222 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.305226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.305312 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.305404 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:55.307162 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.308486 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.311308 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.312134 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.312307 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.314990 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:55.314997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:55.315002 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:55.315045 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:55.318987 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.319944 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.319951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.319955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.320115 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.320660 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.320663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.320665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.321083 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:55.321255 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:55.321692 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 01:06:55.321933 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:06:55.322397 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.299032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.66:38253)
0806 01:06:55.304834 (+ 5802us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.304865 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.305092 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.305494 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.305499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.306171 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:55.306959 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.306985 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.308683 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.308692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.311017 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.311043 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.312364 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.312371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.312544 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.313283 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:55.313312 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:55.314762 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:55.319162 (+ 4400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.319168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.319172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.319809 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.320270 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.320276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.320279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.320448 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.321228 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:55.321233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:55.321430 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 01:06:55.321586 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:06:55.321947 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5547,"thread_start_us":5324,"threads_started":1}
I20260806 01:06:55.324146 24416 ts_manager.cc:194] Registered new tserver with Master: 8e662941504842acaee5e5c488170dec (127.22.63.66:39411)
I20260806 01:06:55.325215 24416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:34109
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:06:55.371673 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40477
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:55.718297 24936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:55.718644 24936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:55.718820 24936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:55.726537 24936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:55.726717 24936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:55.726886 24936 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:06:55.727026 24936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:06:55.737834 24936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:55.739270 24936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:55.740698 24936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:55.760637 24950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:55.761188 24947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:55.761464 24948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:55.762322 24936 server_base.cc:1034] running on GCE node
I20260806 01:06:55.763082 24936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:55.764284 24936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:55.765511 24936 hybrid_clock.cc:648] HybridClock initialized: now 1785978415765419 us; error 54 us; skew 500 ppm
I20260806 01:06:55.765691 24722 heartbeater.cc:503] Master 127.22.63.126:40477 was elected leader, sending a full tablet report...
I20260806 01:06:55.765877 24936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:55.774290 24936 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:06:55.776082 24936 webserver.cc:460] Webserver started at http://127.22.63.67:38463/ using document root <none> and password file <none>
I20260806 01:06:55.776880 24936 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:55.777045 24936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:55.777347 24936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:55.780514 24936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "adc248842f864370a9d44db8150b8298"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "3bbf1a675c8e699cacb9ab26fa88a0a9"
server_key_iv: "9feb77243ed6443450709c2ab9a3ce67"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.781311 24936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "adc248842f864370a9d44db8150b8298"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "3bbf1a675c8e699cacb9ab26fa88a0a9"
server_key_iv: "9feb77243ed6443450709c2ab9a3ce67"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.786824 24936 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:06:55.790568 24959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:55.792129 24936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:06:55.792321 24936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "adc248842f864370a9d44db8150b8298"
format_stamp: "Formatted at 2026-08-06 01:06:55 on dist-test-slave-7f01"
server_key: "3bbf1a675c8e699cacb9ab26fa88a0a9"
server_key_iv: "9feb77243ed6443450709c2ab9a3ce67"
server_key_version: "encryptionkey@0"
I20260806 01:06:55.792573 24936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:55.819272 24936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:55.824177 24936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:55.824524 24936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:55.827500 24936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:55.827617 24936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:55.827824 24936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:55.827909 24936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:55.853065 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.829465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:41795 (local address 127.22.63.126:40477)
0806 01:06:55.830556 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.830563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.830587 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:06:55.832205 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.832211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.832300 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.832433 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:55.835090 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.836252 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.839847 (+ 3595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.840426 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.840573 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.845675 (+ 5102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:55.845680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:55.845683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:55.845719 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:55.848857 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.850011 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.850017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.850023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.850230 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.850928 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.850932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.850934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.851210 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:55.851324 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:55.852544 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0806 01:06:55.852724 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":958,"thread_start_us":846,"threads_started":1}
I20260806 01:06:55.855526 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.828838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.67:41795)
0806 01:06:55.829722 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.829782 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.831922 (+ 2140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.832762 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.832778 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.833780 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:55.834693 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.834831 (+ 138us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.836505 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.836513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.839594 (+ 3081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.839616 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.840748 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.840758 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.841141 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.842073 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:55.842097 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:55.845423 (+ 3326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:55.849031 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.849041 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.849062 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.849865 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.850385 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.850393 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.850400 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.850760 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.851352 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:55.851365 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:55.852044 (+ 679us) client_negotiation.cc:769] Sending connection context
0806 01:06:55.852451 (+ 407us) client_negotiation.cc:240] Negotiation successful
0806 01:06:55.853211 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":367,"threads_started":1}
I20260806 01:06:55.891840 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.870928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:47683 (local address 127.22.63.126:40477)
0806 01:06:55.871193 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.871202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.872016 (+ 814us) server_negotiation.cc:407] Connection header received
0806 01:06:55.872346 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.872354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.872453 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.872599 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:55.874292 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.875636 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.878405 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.879907 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.880189 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.883685 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:55.883696 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:55.883705 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:55.883784 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:55.888422 (+ 4638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.889500 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.889509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.889516 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.889723 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.890156 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.890164 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.890169 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.890593 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:55.890777 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:55.891177 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 01:06:55.891420 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:06:55.892041 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.870996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.22.63.67:47683)
0806 01:06:55.871881 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.871910 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.872124 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.872630 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.872647 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.873434 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:55.874068 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.874093 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.875843 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.875848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.878169 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.878191 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.880222 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.880229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:55.880422 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.881970 (+ 1548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:55.882010 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:55.883273 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:55.888670 (+ 5397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.888682 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.888688 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.889331 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.889864 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:55.889867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:55.889870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:55.890001 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:55.890728 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:55.890733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:55.890928 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 01:06:55.891081 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 01:06:55.891471 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":480,"threads_started":1}
I20260806 01:06:55.892869 24936 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:43505
I20260806 01:06:55.892892 25156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:43505 every 8 connection(s)
I20260806 01:06:55.894300 24936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:06:55.896330 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24936
I20260806 01:06:55.896554 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:06:55.897122 22781 external_mini_cluster.cc:1428] Setting key 1195304d76a443b68693810cd0a28a83
I20260806 01:06:55.898640 25157 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40477
I20260806 01:06:55.898941 25157 heartbeater.cc:464] Registering TS with master...
I20260806 01:06:55.900072 25157 heartbeater.cc:511] Master 127.22.63.126:40477 requested a full tablet report, sending...
I20260806 01:06:55.901737 24417 ts_manager.cc:194] Registered new tserver with Master: adc248842f864370a9d44db8150b8298 (127.22.63.67:43505)
I20260806 01:06:55.902539 24417 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:41795
I20260806 01:06:55.912923 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:06:55 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:55.968549 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.953760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.22.63.126:40477)
0806 01:06:55.953940 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.953945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.954314 (+ 369us) server_negotiation.cc:407] Connection header received
0806 01:06:55.954518 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.954525 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.954610 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.954702 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:55.956158 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.956877 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.958258 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:55.958762 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:55.958877 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:55.962859 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:55.962864 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:55.962867 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:55.962902 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:55.965610 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.966410 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.966417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.966421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.966563 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:55.966997 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:55.967003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:55.967006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:55.967305 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:55.967578 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:55.967936 (+ 358us) server_negotiation.cc:299] Negotiation successful
0806 01:06:55.968167 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:55.973068 24417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:06:55.980284 24417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:06:56.009960 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.992952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39411 (local address 127.0.0.1:54016)
0806 01:06:55.993800 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.993845 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.994044 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.998155 (+ 4111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.998164 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.998197 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:55.998756 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.998771 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.001090 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.001096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.003966 (+ 2870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.003983 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.009133 (+ 5150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.009138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.009337 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.009367 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.009425 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.009533 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":21,"thread_start_us":347,"threads_started":1}
I20260806 01:06:56.010505 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.993271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54016 (local address 127.22.63.66:39411)
0806 01:06:55.996914 (+ 3643us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.996924 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.996961 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:06:55.997093 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.997102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.997870 (+ 768us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.999032 (+ 1162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:55.999259 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.000859 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.007268 (+ 6409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.008859 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.009691 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.009916 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.010012 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.010208 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3463,"thread_start_us":3325,"threads_started":1}
I20260806 01:06:56.011087 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.993976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.22.63.67:43505)
0806 01:06:55.996438 (+ 2462us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:55.996451 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:55.997666 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 01:06:55.997790 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:55.997797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:55.997985 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:55.998178 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:55.999275 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.000932 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.006532 (+ 5600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.008159 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.008302 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.008582 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.010534 (+ 1952us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.010720 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2225,"thread_start_us":2012,"threads_started":1}
I20260806 01:06:56.011101 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.993267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43505 (local address 127.0.0.1:40064)
0806 01:06:55.997237 (+ 3970us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:55.997264 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:55.997526 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:55.998531 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:55.998538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:55.998569 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:55.999054 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:55.999066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.001097 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.001101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.006291 (+ 5190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.006310 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.008748 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.008756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.009004 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.009086 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.009239 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.009465 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3820,"mutex_wait_us":4,"thread_start_us":3681,"threads_started":1}
I20260806 01:06:56.014472 24825 tablet_service.cc:1467] Processing CreateTablet for tablet 96e8b1675e904c10af04e0faba813bd4 (DEFAULT_TABLE table=test-table [id=49e1e0358bd041c1b6c1ccab5021507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.014884 25048 tablet_service.cc:1467] Processing CreateTablet for tablet 96e8b1675e904c10af04e0faba813bd4 (DEFAULT_TABLE table=test-table [id=49e1e0358bd041c1b6c1ccab5021507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.015923 24825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e8b1675e904c10af04e0faba813bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.016271 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e8b1675e904c10af04e0faba813bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.019577 25180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.997622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45361 (local address 127.0.0.1:43410)
0806 01:06:56.010525 (+ 12903us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.010546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.010770 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.011829 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.011833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.011859 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:56.012311 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.012324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.014290 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.014295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.016371 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.016388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.018671 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.018678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.018873 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.018914 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.019009 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.019153 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12749,"thread_start_us":12600,"threads_started":1}
I20260806 01:06:56.019609 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:55.997655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.22.63.65:45361)
0806 01:06:56.009774 (+ 12119us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.009787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.010652 (+ 865us) server_negotiation.cc:407] Connection header received
0806 01:06:56.010999 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.011009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.011325 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.011514 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:56.012522 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.014057 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.016578 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.018110 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.018265 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.018476 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.019119 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.019294 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11865,"thread_start_us":12147,"threads_started":1}
I20260806 01:06:56.023916 24624 tablet_service.cc:1467] Processing CreateTablet for tablet 96e8b1675e904c10af04e0faba813bd4 (DEFAULT_TABLE table=test-table [id=49e1e0358bd041c1b6c1ccab5021507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.026580 24624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e8b1675e904c10af04e0faba813bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.029096 25183 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Bootstrap starting.
I20260806 01:06:56.032640 25183 tablet_bootstrap.cc:654] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.032620 25184 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Bootstrap starting.
I20260806 01:06:56.034229 25183 log.cc:824] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:56.036882 25184 tablet_bootstrap.cc:654] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.038115 25184 log.cc:824] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:56.039903 25183 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: No bootstrap required, opened a new log
I20260806 01:06:56.040215 25183 ts_tablet_manager.cc:1397] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260806 01:06:56.041388 25187 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Bootstrap starting.
I20260806 01:06:56.041435 25184 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: No bootstrap required, opened a new log
I20260806 01:06:56.041707 25184 ts_tablet_manager.cc:1397] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.005s
I20260806 01:06:56.044514 25187 tablet_bootstrap.cc:654] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.046160 25187 log.cc:824] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:56.050089 25187 tablet_bootstrap.cc:492] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: No bootstrap required, opened a new log
I20260806 01:06:56.050407 25187 ts_tablet_manager.cc:1397] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260806 01:06:56.051759 25183 raft_consensus.cc:348] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.052228 25183 raft_consensus.cc:374] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.052347 25183 raft_consensus.cc:729] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e662941504842acaee5e5c488170dec, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.052613 25184 raft_consensus.cc:348] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.053042 25184 raft_consensus.cc:374] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.053141 25184 raft_consensus.cc:729] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc248842f864370a9d44db8150b8298, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.053036 25183 consensus_queue.cc:260] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.054250 24931 heartbeater.cc:503] Master 127.22.63.126:40477 was elected leader, sending a full tablet report...
I20260806 01:06:56.054983 25183 ts_tablet_manager.cc:1428] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260806 01:06:56.053835 25184 consensus_queue.cc:260] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.056526 25157 heartbeater.cc:503] Master 127.22.63.126:40477 was elected leader, sending a full tablet report...
I20260806 01:06:56.058396 25184 ts_tablet_manager.cc:1428] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260806 01:06:56.060923 25187 raft_consensus.cc:348] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.061379 25187 raft_consensus.cc:374] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.061501 25187 raft_consensus.cc:729] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b7ad32f1d84391bd62ff76007e5566, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.062144 25187 consensus_queue.cc:260] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.063606 25187 ts_tablet_manager.cc:1428] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260806 01:06:56.071012 25195 raft_consensus.cc:482] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:06:56.071272 25195 raft_consensus.cc:504] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.072918 25195 leader_election.cc:288] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e662941504842acaee5e5c488170dec (127.22.63.66:39411), 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361)
W20260806 01:06:56.076578 24932 tablet.cc:2369] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:56.079501 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.073169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39411 (local address 127.22.63.67:53719)
0806 01:06:56.073507 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.073534 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.073666 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.074085 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.074092 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.074128 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:56.074375 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.074388 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.075554 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.075560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.077539 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.077552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.078708 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.078714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.078909 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.078934 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.079070 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.079212 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260806 01:06:56.079505 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.073372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:53719 (local address 127.22.63.66:39411)
0806 01:06:56.073584 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.073589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.073610 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:06:56.073846 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.073852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.073939 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.074048 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:56.074538 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.075397 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.077708 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.078542 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.078725 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.078808 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.079097 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.079226 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:06:56.079824 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.073424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:55451 (local address 127.22.63.65:45361)
0806 01:06:56.073697 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.073706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.074234 (+ 528us) server_negotiation.cc:407] Connection header received
0806 01:06:56.074381 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.074385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.074452 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.074568 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:56.075315 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.076322 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.078091 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.078874 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.079052 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.079130 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.079425 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.079557 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260806 01:06:56.079941 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.073316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45361 (local address 127.22.63.67:55451)
0806 01:06:56.074120 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.074137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.074259 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.074583 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.074587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.074614 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:56.074923 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.074932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.076457 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.076463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.077927 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.077939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.079098 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.079104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.079260 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.079284 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.079390 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.079522 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":3,"thread_start_us":447,"threads_started":1}
I20260806 01:06:56.080965 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e8b1675e904c10af04e0faba813bd4" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b7ad32f1d84391bd62ff76007e5566" is_pre_election: true
I20260806 01:06:56.080958 24852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e8b1675e904c10af04e0faba813bd4" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec" is_pre_election: true
I20260806 01:06:56.081459 24657 raft_consensus.cc:2454] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 0.
I20260806 01:06:56.081460 24852 raft_consensus.cc:2454] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 0.
I20260806 01:06:56.082139 24964 leader_election.cc:302] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, 8e662941504842acaee5e5c488170dec, adc248842f864370a9d44db8150b8298; no voters:
I20260806 01:06:56.082566 25195 raft_consensus.cc:2781] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:06:56.082720 25195 raft_consensus.cc:482] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:06:56.082828 25195 raft_consensus.cc:3037] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.085170 25195 raft_consensus.cc:504] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.085844 25195 leader_election.cc:288] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 election: Requested vote from peers 8e662941504842acaee5e5c488170dec (127.22.63.66:39411), 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361)
I20260806 01:06:56.086287 24852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e8b1675e904c10af04e0faba813bd4" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec"
I20260806 01:06:56.086323 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e8b1675e904c10af04e0faba813bd4" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b7ad32f1d84391bd62ff76007e5566"
I20260806 01:06:56.086500 24852 raft_consensus.cc:3037] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.086532 24657 raft_consensus.cc:3037] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.088778 24852 raft_consensus.cc:2454] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 1.
I20260806 01:06:56.088853 24657 raft_consensus.cc:2454] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 1.
I20260806 01:06:56.089306 24964 leader_election.cc:302] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, 8e662941504842acaee5e5c488170dec, adc248842f864370a9d44db8150b8298; no voters:
I20260806 01:06:56.089623 25195 raft_consensus.cc:2781] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:56.090119 25195 raft_consensus.cc:686] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 1 LEADER]: Becoming Leader. State: Replica: adc248842f864370a9d44db8150b8298, State: Running, Role: LEADER
I20260806 01:06:56.090592 25195 consensus_queue.cc:237] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } }
I20260806 01:06:56.095906 24416 catalog_manager.cc:5495] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 reported cstate change: term changed from 0 to 1, leader changed from <none> to adc248842f864370a9d44db8150b8298 (127.22.63.67). New cstate: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } health_report { overall_health: UNKNOWN } } }
I20260806 01:06:56.142850 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.136966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.22.63.67:43505)
0806 01:06:56.137226 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.137232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.137358 (+ 126us) server_negotiation.cc:407] Connection header received
0806 01:06:56.137738 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.137742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.137845 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.138000 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:56.138396 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.139119 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.140555 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.140994 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.141133 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.141657 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.142176 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.142347 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.142478 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.142568 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260806 01:06:56.150962 25158 tablet.cc:2369] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:56.151651 24852 raft_consensus.cc:1261] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Refusing update from remote peer adc248842f864370a9d44db8150b8298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:56.151674 24657 raft_consensus.cc:1261] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Refusing update from remote peer adc248842f864370a9d44db8150b8298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:56.152815 25195 consensus_queue.cc:1035] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.153163 25199 consensus_queue.cc:1035] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.175488 25207 mvcc.cc:205] Tried to move back new op lower bound from 7315367592545566720 to 7315367592311181312. Current Snapshot: MvccSnapshot[applied={T|T < 7315367592545566720 or (T in {7315367592545566720})}]
I20260806 01:06:56.189034 24416 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:06:56.189414 24416 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:06:56.193559 24416 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Sending DeleteTablet for 3 replicas of tablet 96e8b1675e904c10af04e0faba813bd4
I20260806 01:06:56.195006 25048 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e8b1675e904c10af04e0faba813bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40064
I20260806 01:06:56.194972 24825 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e8b1675e904c10af04e0faba813bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:54016
I20260806 01:06:56.195281 24624 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e8b1675e904c10af04e0faba813bd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:43410
I20260806 01:06:56.200541 24416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:06:56.201694 24416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:06:56.204607 25218 tablet_replica.cc:330] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: stopping tablet replica
I20260806 01:06:56.206981 25218 raft_consensus.cc:2229] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:06:56.207930 25218 raft_consensus.cc:2258] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.209973 25217 tablet_replica.cc:330] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: stopping tablet replica
I20260806 01:06:56.210435 25218 ts_tablet_manager.cc:1905] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:56.210579 25217 raft_consensus.cc:2229] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:06:56.211158 25217 raft_consensus.cc:2258] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.212425 25048 tablet_service.cc:1467] Processing CreateTablet for tablet cb05de7e57f047b0a328d27b951dc256 (DEFAULT_TABLE table=test-table [id=a78c52d8386d4da7899d12d7d0d844f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.212491 24621 tablet_service.cc:1467] Processing CreateTablet for tablet cb05de7e57f047b0a328d27b951dc256 (DEFAULT_TABLE table=test-table [id=a78c52d8386d4da7899d12d7d0d844f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.212977 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb05de7e57f047b0a328d27b951dc256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.213030 24621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb05de7e57f047b0a328d27b951dc256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.218910 24825 tablet_service.cc:1467] Processing CreateTablet for tablet cb05de7e57f047b0a328d27b951dc256 (DEFAULT_TABLE table=test-table [id=a78c52d8386d4da7899d12d7d0d844f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:06:56.219461 24825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb05de7e57f047b0a328d27b951dc256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.220770 25217 ts_tablet_manager.cc:1905] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:56.221206 25187 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Bootstrap starting.
I20260806 01:06:56.222927 25187 tablet_bootstrap.cc:654] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.224766 25222 tablet_replica.cc:330] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: stopping tablet replica
I20260806 01:06:56.225193 25222 raft_consensus.cc:2229] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:06:56.225718 25222 raft_consensus.cc:2258] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.228070 25222 ts_tablet_manager.cc:1905] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:56.228354 25184 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Bootstrap starting.
I20260806 01:06:56.231508 25218 ts_tablet_manager.cc:1918] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:06:56.231685 25218 log.cc:1196] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/96e8b1675e904c10af04e0faba813bd4
I20260806 01:06:56.232023 25183 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Bootstrap starting.
I20260806 01:06:56.232316 25218 ts_tablet_manager.cc:1939] T 96e8b1675e904c10af04e0faba813bd4 P adc248842f864370a9d44db8150b8298: Deleting consensus metadata
I20260806 01:06:56.233682 25183 tablet_bootstrap.cc:654] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.233889 25184 tablet_bootstrap.cc:654] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.234301 24397 catalog_manager.cc:4841] TS adc248842f864370a9d44db8150b8298 (127.22.63.67:43505): tablet 96e8b1675e904c10af04e0faba813bd4 (table test-table [id=49e1e0358bd041c1b6c1ccab5021507b]) successfully deleted
I20260806 01:06:56.238128 25187 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: No bootstrap required, opened a new log
I20260806 01:06:56.238320 25187 ts_tablet_manager.cc:1397] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.004s
I20260806 01:06:56.239380 25187 raft_consensus.cc:348] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.239637 25187 raft_consensus.cc:374] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.239781 25187 raft_consensus.cc:729] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b7ad32f1d84391bd62ff76007e5566, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.240185 25187 consensus_queue.cc:260] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.240882 25187 ts_tablet_manager.cc:1428] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 01:06:56.242210 25217 ts_tablet_manager.cc:1918] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:06:56.242393 25217 log.cc:1196] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/96e8b1675e904c10af04e0faba813bd4
I20260806 01:06:56.242674 25184 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: No bootstrap required, opened a new log
I20260806 01:06:56.242838 25184 ts_tablet_manager.cc:1397] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.006s
I20260806 01:06:56.243013 25217 ts_tablet_manager.cc:1939] T 96e8b1675e904c10af04e0faba813bd4 P 8e662941504842acaee5e5c488170dec: Deleting consensus metadata
I20260806 01:06:56.243455 25222 ts_tablet_manager.cc:1918] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:06:56.243683 25222 log.cc:1196] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/96e8b1675e904c10af04e0faba813bd4
I20260806 01:06:56.243696 25184 raft_consensus.cc:348] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.244016 25184 raft_consensus.cc:374] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.244117 25183 tablet_bootstrap.cc:492] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: No bootstrap required, opened a new log
I20260806 01:06:56.244158 25184 raft_consensus.cc:729] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc248842f864370a9d44db8150b8298, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.244309 25183 ts_tablet_manager.cc:1397] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260806 01:06:56.244475 25222 ts_tablet_manager.cc:1939] T 96e8b1675e904c10af04e0faba813bd4 P 59b7ad32f1d84391bd62ff76007e5566: Deleting consensus metadata
I20260806 01:06:56.244495 25184 consensus_queue.cc:260] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.245163 25184 ts_tablet_manager.cc:1428] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260806 01:06:56.245157 24398 catalog_manager.cc:4841] TS 8e662941504842acaee5e5c488170dec (127.22.63.66:39411): tablet 96e8b1675e904c10af04e0faba813bd4 (table test-table [id=49e1e0358bd041c1b6c1ccab5021507b]) successfully deleted
I20260806 01:06:56.245703 25183 raft_consensus.cc:348] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.246006 25183 raft_consensus.cc:374] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.246821 25183 raft_consensus.cc:729] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e662941504842acaee5e5c488170dec, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.247169 25183 consensus_queue.cc:260] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.247831 25183 ts_tablet_manager.cc:1428] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:06:56.250231 24397 catalog_manager.cc:4841] TS 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361): tablet 96e8b1675e904c10af04e0faba813bd4 (table test-table [id=49e1e0358bd041c1b6c1ccab5021507b]) successfully deleted
W20260806 01:06:56.258664 24723 tablet.cc:2369] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:06:56.262506 25196 raft_consensus.cc:482] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:06:56.262719 25196 raft_consensus.cc:504] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.264731 25196 leader_election.cc:288] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e662941504842acaee5e5c488170dec (127.22.63.66:39411), adc248842f864370a9d44db8150b8298 (127.22.63.67:43505)
I20260806 01:06:56.275823 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.265403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:44501 (local address 127.22.63.66:39411)
0806 01:06:56.265644 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.265653 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.266200 (+ 547us) server_negotiation.cc:407] Connection header received
0806 01:06:56.266399 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.266408 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.266509 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.266603 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:56.267422 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.268452 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.272263 (+ 3811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.273920 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.274157 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.274256 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.275029 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.275219 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:06:56.276209 25231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.264935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39411 (local address 127.22.63.65:44501)
0806 01:06:56.266074 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.266095 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.266241 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.266811 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.266815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.266837 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:56.267237 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.267253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.268602 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.268608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.272063 (+ 3455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.272085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.274422 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.274432 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.274747 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.274803 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.274995 (+ 192us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.275165 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":171,"thread_start_us":1532,"threads_started":1}
I20260806 01:06:56.276264 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.265357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:39367 (local address 127.22.63.67:43505)
0806 01:06:56.265549 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.265556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.266926 (+ 1370us) server_negotiation.cc:407] Connection header received
0806 01:06:56.267108 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.267121 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.267207 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.267324 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:06:56.267994 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.268959 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.272464 (+ 3505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.274566 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.274776 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.274906 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.275492 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.275648 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:06:56.277659 25229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.264980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43505 (local address 127.22.63.65:39367)
0806 01:06:56.266794 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.266815 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.266953 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.267363 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.267368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.267395 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:06:56.267811 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.267823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.269091 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.269103 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.272197 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.272216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.274977 (+ 2761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.274987 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.275245 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.275278 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.276453 (+ 1175us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.276632 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1662,"mutex_wait_us":1,"thread_start_us":808,"threads_started":1}
I20260806 01:06:56.277747 24851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb05de7e57f047b0a328d27b951dc256" candidate_uuid: "59b7ad32f1d84391bd62ff76007e5566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec" is_pre_election: true
I20260806 01:06:56.278496 24851 raft_consensus.cc:2454] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59b7ad32f1d84391bd62ff76007e5566 in term 0.
I20260806 01:06:56.279347 24553 leader_election.cc:302] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, 8e662941504842acaee5e5c488170dec; no voters:
I20260806 01:06:56.280205 25196 raft_consensus.cc:2781] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:06:56.279592 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb05de7e57f047b0a328d27b951dc256" candidate_uuid: "59b7ad32f1d84391bd62ff76007e5566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc248842f864370a9d44db8150b8298" is_pre_election: true
I20260806 01:06:56.280511 25196 raft_consensus.cc:482] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:06:56.280804 25196 raft_consensus.cc:3037] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.280810 25083 raft_consensus.cc:2454] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59b7ad32f1d84391bd62ff76007e5566 in term 0.
I20260806 01:06:56.285125 25196 raft_consensus.cc:504] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.286082 25196 leader_election.cc:288] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [CANDIDATE]: Term 1 election: Requested vote from peers 8e662941504842acaee5e5c488170dec (127.22.63.66:39411), adc248842f864370a9d44db8150b8298 (127.22.63.67:43505)
I20260806 01:06:56.286934 25083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb05de7e57f047b0a328d27b951dc256" candidate_uuid: "59b7ad32f1d84391bd62ff76007e5566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc248842f864370a9d44db8150b8298"
I20260806 01:06:56.286962 24851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb05de7e57f047b0a328d27b951dc256" candidate_uuid: "59b7ad32f1d84391bd62ff76007e5566" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec"
I20260806 01:06:56.287307 24851 raft_consensus.cc:3037] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.287307 25083 raft_consensus.cc:3037] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.292862 24851 raft_consensus.cc:2454] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59b7ad32f1d84391bd62ff76007e5566 in term 1.
I20260806 01:06:56.293565 24553 leader_election.cc:302] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, 8e662941504842acaee5e5c488170dec; no voters:
I20260806 01:06:56.293758 25083 raft_consensus.cc:2454] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59b7ad32f1d84391bd62ff76007e5566 in term 1.
I20260806 01:06:56.293991 25196 raft_consensus.cc:2781] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:56.295059 25196 raft_consensus.cc:686] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 LEADER]: Becoming Leader. State: Replica: 59b7ad32f1d84391bd62ff76007e5566, State: Running, Role: LEADER
I20260806 01:06:56.295831 25196 consensus_queue.cc:237] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.302090 24416 catalog_manager.cc:5495] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "59b7ad32f1d84391bd62ff76007e5566" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:06:56 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978413, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:56.379958 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.356243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.22.63.126:40477)
0806 01:06:56.356420 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.356428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.358534 (+ 2106us) server_negotiation.cc:407] Connection header received
0806 01:06:56.358816 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.358827 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.358886 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.359036 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:56.361454 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.362429 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.364462 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.365199 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.365350 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.371065 (+ 5715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:56.371073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:56.371078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:56.371121 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:56.375582 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:56.377157 (+ 1575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:56.377161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:56.377163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:56.377381 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:56.378054 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:56.378058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:56.378060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:56.378426 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:56.378549 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.379261 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.379527 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:06:56.380728 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.356848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40477 (local address 127.0.0.1:49560)
0806 01:06:56.358418 (+ 1570us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.358436 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.358677 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.359022 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.359027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.360512 (+ 1485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:56.361296 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.361310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.362571 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.362578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.364308 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.364322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.365390 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.365397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.365600 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.366412 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:56.366431 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:56.370742 (+ 4311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:56.375831 (+ 5089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:56.375851 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:56.375873 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:56.376989 (+ 1116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:56.377511 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:56.377517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:56.377521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:56.377873 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:56.378581 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:56.378592 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:56.378934 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.379170 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.379788 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":193,"thread_start_us":1047,"threads_started":1}
I20260806 01:06:56.388864 24416 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:49560:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:06:56.390899 24416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:06:56.400287 24621 tablet_service.cc:1467] Processing CreateTablet for tablet 34d38f0461ee428abc9c12a25edaa465 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4a6b9749c6b243dcbdc9ba4e56d0bdef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:06:56.400899 24621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d38f0461ee428abc9c12a25edaa465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.404490 25048 tablet_service.cc:1467] Processing CreateTablet for tablet 34d38f0461ee428abc9c12a25edaa465 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4a6b9749c6b243dcbdc9ba4e56d0bdef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:06:56.405932 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d38f0461ee428abc9c12a25edaa465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.406571 24825 tablet_service.cc:1467] Processing CreateTablet for tablet 34d38f0461ee428abc9c12a25edaa465 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4a6b9749c6b243dcbdc9ba4e56d0bdef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:06:56.407231 24825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34d38f0461ee428abc9c12a25edaa465. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:56.416077 25183 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: Bootstrap starting.
I20260806 01:06:56.418394 25184 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: Bootstrap starting.
I20260806 01:06:56.418987 25187 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: Bootstrap starting.
I20260806 01:06:56.419353 25183 tablet_bootstrap.cc:654] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.420852 25184 tablet_bootstrap.cc:654] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.423635 25187 tablet_bootstrap.cc:654] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:56.426087 25184 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: No bootstrap required, opened a new log
I20260806 01:06:56.426286 25184 ts_tablet_manager.cc:1397] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260806 01:06:56.426762 25183 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: No bootstrap required, opened a new log
I20260806 01:06:56.426976 25183 ts_tablet_manager.cc:1397] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260806 01:06:56.427141 25184 raft_consensus.cc:348] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.427436 25184 raft_consensus.cc:374] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.427528 25184 raft_consensus.cc:729] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc248842f864370a9d44db8150b8298, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.427819 25184 consensus_queue.cc:260] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.427963 25183 raft_consensus.cc:348] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.428275 25183 raft_consensus.cc:374] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.428373 25183 raft_consensus.cc:729] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e662941504842acaee5e5c488170dec, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.428488 25184 ts_tablet_manager.cc:1428] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:06:56.428716 25183 consensus_queue.cc:260] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.429316 25193 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.429701 25193 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.430116 25213 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.430480 25213 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.430787 25183 ts_tablet_manager.cc:1428] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:06:56.437059 25187 tablet_bootstrap.cc:492] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: No bootstrap required, opened a new log
I20260806 01:06:56.437263 25187 ts_tablet_manager.cc:1397] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.006s
I20260806 01:06:56.438158 25187 raft_consensus.cc:348] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.438464 25187 raft_consensus.cc:374] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:56.438555 25187 raft_consensus.cc:729] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b7ad32f1d84391bd62ff76007e5566, State: Initialized, Role: FOLLOWER
I20260806 01:06:56.438822 25187 consensus_queue.cc:260] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.439311 25196 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.439641 25196 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.440237 25187 ts_tablet_manager.cc:1428] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 01:06:56.638927 25213 raft_consensus.cc:482] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:06:56.639245 25213 raft_consensus.cc:504] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.640270 25213 leader_election.cc:288] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361), 8e662941504842acaee5e5c488170dec (127.22.63.66:39411)
I20260806 01:06:56.640882 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d38f0461ee428abc9c12a25edaa465" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b7ad32f1d84391bd62ff76007e5566" is_pre_election: true
I20260806 01:06:56.640918 24851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d38f0461ee428abc9c12a25edaa465" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec" is_pre_election: true
I20260806 01:06:56.641146 24657 raft_consensus.cc:2454] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 0.
I20260806 01:06:56.641248 24851 raft_consensus.cc:2454] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 0.
I20260806 01:06:56.641647 24961 leader_election.cc:302] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, adc248842f864370a9d44db8150b8298; no voters:
I20260806 01:06:56.641973 25213 raft_consensus.cc:2781] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:06:56.642092 25213 raft_consensus.cc:482] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:06:56.642220 25213 raft_consensus.cc:3037] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.644744 25213 raft_consensus.cc:504] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.645356 25213 leader_election.cc:288] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 election: Requested vote from peers 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361), 8e662941504842acaee5e5c488170dec (127.22.63.66:39411)
I20260806 01:06:56.645922 24851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d38f0461ee428abc9c12a25edaa465" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e662941504842acaee5e5c488170dec"
I20260806 01:06:56.645886 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34d38f0461ee428abc9c12a25edaa465" candidate_uuid: "adc248842f864370a9d44db8150b8298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b7ad32f1d84391bd62ff76007e5566"
I20260806 01:06:56.646148 24657 raft_consensus.cc:3037] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.646168 24851 raft_consensus.cc:3037] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:56.649183 24851 raft_consensus.cc:2454] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 1.
I20260806 01:06:56.649201 24657 raft_consensus.cc:2454] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adc248842f864370a9d44db8150b8298 in term 1.
I20260806 01:06:56.649667 24961 leader_election.cc:302] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b7ad32f1d84391bd62ff76007e5566, adc248842f864370a9d44db8150b8298; no voters:
I20260806 01:06:56.650182 25213 raft_consensus.cc:2781] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:56.650655 25213 raft_consensus.cc:686] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [term 1 LEADER]: Becoming Leader. State: Replica: adc248842f864370a9d44db8150b8298, State: Running, Role: LEADER
I20260806 01:06:56.651001 25213 consensus_queue.cc:237] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } }
I20260806 01:06:56.653110 25195 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: TxnStatusTablet state changed. Reason: New leader adc248842f864370a9d44db8150b8298. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.653435 25195 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:06:56.654258 25247 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:06:56.654798 25247 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:06:56.655396 24416 catalog_manager.cc:5495] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 reported cstate change: term changed from 0 to 1, leader changed from <none> to adc248842f864370a9d44db8150b8298 (127.22.63.67). New cstate: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } health_report { overall_health: HEALTHY } } }
I20260806 01:06:56.720960 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.715867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.22.63.67:43505)
0806 01:06:56.716075 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.716082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.716174 (+ 92us) server_negotiation.cc:407] Connection header received
0806 01:06:56.716461 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.716467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.716555 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.716657 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:56.717052 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.717790 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.719108 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.719549 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.719697 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.720186 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.720291 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.720396 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.720565 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.720645 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:06:56.720968 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.715780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43505 (local address 127.0.0.1:40082)
0806 01:06:56.716059 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.716082 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.716270 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.716690 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.716695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.716707 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:06:56.716909 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.716918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.717925 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.717933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.718944 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.718956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.719714 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.719719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.719924 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.719985 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.720398 (+ 413us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.720432 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.720538 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.720691 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:06:56.725667 24851 raft_consensus.cc:1261] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Refusing update from remote peer adc248842f864370a9d44db8150b8298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:56.725716 24657 raft_consensus.cc:1261] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Refusing update from remote peer adc248842f864370a9d44db8150b8298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:06:56.726353 25195 consensus_queue.cc:1035] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.726890 25213 consensus_queue.cc:1035] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.729562 25196 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: TxnStatusTablet state changed. Reason: New leader adc248842f864370a9d44db8150b8298. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.729851 25196 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.734568 25196 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.734915 25196 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 59b7ad32f1d84391bd62ff76007e5566: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.738009 25213 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.738332 25213 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:06:56.739853 25249 mvcc.cc:205] Tried to move back new op lower bound from 7315367594902278144 to 7315367594605756416. Current Snapshot: MvccSnapshot[applied={T|T < 7315367594902278144}]
I20260806 01:06:56.749611 25193 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: TxnStatusTablet state changed. Reason: New leader adc248842f864370a9d44db8150b8298. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.750075 25193 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.753078 25213 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.753371 25213 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P adc248842f864370a9d44db8150b8298: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:06:56.761044 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.749625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.22.63.65:45361)
0806 01:06:56.752480 (+ 2855us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.752488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.752529 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:06:56.752648 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.752652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.752767 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.753067 (+ 300us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:56.753583 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.755011 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.757384 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.758221 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.758338 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.759066 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.759915 (+ 849us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.760056 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.760587 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.760674 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2730,"thread_start_us":7306,"threads_started":1}
I20260806 01:06:56.775545 25193 tablet_replica.cc:439] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "adc248842f864370a9d44db8150b8298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b7ad32f1d84391bd62ff76007e5566" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45361 } } peers { permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 } } peers { permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 } } }
I20260806 01:06:56.775898 25193 tablet_replica.cc:442] T 34d38f0461ee428abc9c12a25edaa465 P 8e662941504842acaee5e5c488170dec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:06:56.782405 25196 consensus_queue.cc:1035] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e662941504842acaee5e5c488170dec" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.802838 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.782902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:43347 (local address 127.22.63.67:43505)
0806 01:06:56.783128 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.783134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.793792 (+ 10658us) server_negotiation.cc:407] Connection header received
0806 01:06:56.794338 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.794345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.794433 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.794503 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:56.795222 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.796468 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.800527 (+ 4059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.801234 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.801330 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.801968 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.802133 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.802192 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.802406 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.802504 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:06:56.803570 25269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.782441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43505 (local address 127.22.63.65:43347)
0806 01:06:56.793673 (+ 11232us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.793695 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.794145 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.794701 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.794706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.794735 (+ 29us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:06:56.795002 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.795062 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.798477 (+ 3415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.798483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.800341 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.800357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.801519 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.801525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.801696 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.801769 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.802272 (+ 503us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.802282 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.803136 (+ 854us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.803331 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10999,"thread_start_us":11370,"threads_started":1}
I20260806 01:06:56.807508 25271 consensus_queue.cc:1035] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc248842f864370a9d44db8150b8298" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:06:56.852751 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.845108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:38999 (local address 127.22.63.65:45361)
0806 01:06:56.845293 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.845299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.846343 (+ 1044us) server_negotiation.cc:407] Connection header received
0806 01:06:56.846470 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.846477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.846576 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.846680 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:06:56.847190 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.848538 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.850486 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.851256 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.851405 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.851898 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.852005 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.852114 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:56.852310 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 01:06:56.852411 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:06:56.852877 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.845170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45361 (local address 127.22.63.67:38999)
0806 01:06:56.846212 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:56.846245 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:56.846362 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:56.846715 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:56.846718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:56.846728 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:06:56.847037 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.847045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.848692 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.848698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.850335 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:56.850349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:56.851443 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:56.851448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:56.851574 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:56.851637 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:06:56.852122 (+ 485us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:06:56.852137 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 01:06:56.852233 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:06:56.852400 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":611,"threads_started":1}
I20260806 01:06:56.897260 24417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:06:56.897571 24417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:06:56.901563 24417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7ef5acb21c54d0e86ea1c131f7985ff: Sending DeleteTablet for 3 replicas of tablet cb05de7e57f047b0a328d27b951dc256
I20260806 01:06:56.902863 24621 tablet_service.cc:1514] Processing DeleteTablet for tablet cb05de7e57f047b0a328d27b951dc256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:43410
I20260806 01:06:56.903383 25048 tablet_service.cc:1514] Processing DeleteTablet for tablet cb05de7e57f047b0a328d27b951dc256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40064
I20260806 01:06:56.903851 24825 tablet_service.cc:1514] Processing DeleteTablet for tablet cb05de7e57f047b0a328d27b951dc256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:06:56 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:54016
I20260806 01:06:56.955650 25285 tablet_replica.cc:330] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: stopping tablet replica
I20260806 01:06:56.956048 25285 raft_consensus.cc:2229] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:06:56.957981 25284 tablet_replica.cc:330] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: stopping tablet replica
I20260806 01:06:56.958348 25284 raft_consensus.cc:2229] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:06:56.958767 25284 raft_consensus.cc:2258] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.960150 25284 ts_tablet_manager.cc:1905] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:56.964035 25285 raft_consensus.cc:2258] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.970721 25285 ts_tablet_manager.cc:1905] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:56.976262 25282 tablet_replica.cc:330] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: stopping tablet replica
I20260806 01:06:56.976579 25282 raft_consensus.cc:2229] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:06:56.977339 25282 raft_consensus.cc:2258] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:06:56.984584 25282 ts_tablet_manager.cc:1905] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:06:57.000290 25285 ts_tablet_manager.cc:1918] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:06:57.000463 25285 log.cc:1196] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/cb05de7e57f047b0a328d27b951dc256
I20260806 01:06:57.000923 25285 ts_tablet_manager.cc:1939] T cb05de7e57f047b0a328d27b951dc256 P 8e662941504842acaee5e5c488170dec: Deleting consensus metadata
I20260806 01:06:57.002159 24398 catalog_manager.cc:4841] TS 8e662941504842acaee5e5c488170dec (127.22.63.66:39411): tablet cb05de7e57f047b0a328d27b951dc256 (table test-table [id=a78c52d8386d4da7899d12d7d0d844f5]) successfully deleted
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:06:57.019917 25284 ts_tablet_manager.cc:1918] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:06:57.021181 25282 ts_tablet_manager.cc:1918] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:06:57.026835 25284 log.cc:1196] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/cb05de7e57f047b0a328d27b951dc256
I20260806 01:06:57.027518 25284 ts_tablet_manager.cc:1939] T cb05de7e57f047b0a328d27b951dc256 P adc248842f864370a9d44db8150b8298: Deleting consensus metadata
I20260806 01:06:57.029894 25282 log.cc:1196] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/cb05de7e57f047b0a328d27b951dc256
I20260806 01:06:57.031713 25282 ts_tablet_manager.cc:1939] T cb05de7e57f047b0a328d27b951dc256 P 59b7ad32f1d84391bd62ff76007e5566: Deleting consensus metadata
I20260806 01:06:57.032740 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:56.920295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.22.63.65:45361)
0806 01:06:56.920540 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:56.920546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:56.997231 (+ 76685us) server_negotiation.cc:407] Connection header received
0806 01:06:56.997637 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:56.997643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:56.997765 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:56.997864 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:57.000108 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.001330 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.003609 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.004336 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.004445 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:57.014265 (+ 9820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:57.014286 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:57.014307 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:57.014358 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:57.018712 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.027781 (+ 9069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.027802 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.027806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.027984 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.028649 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.028655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.028659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.029197 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:57.029544 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:57.032086 (+ 2542us) server_negotiation.cc:299] Negotiation successful
0806 01:06:57.032352 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 01:06:57.034613 24713 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43430
I20260806 01:06:57.035491 24397 catalog_manager.cc:4841] TS 59b7ad32f1d84391bd62ff76007e5566 (127.22.63.65:45361): tablet cb05de7e57f047b0a328d27b951dc256 (table test-table [id=a78c52d8386d4da7899d12d7d0d844f5]) successfully deleted
I20260806 01:06:57.036280 24397 catalog_manager.cc:4841] TS adc248842f864370a9d44db8150b8298 (127.22.63.67:43505): tablet cb05de7e57f047b0a328d27b951dc256 (table test-table [id=a78c52d8386d4da7899d12d7d0d844f5]) successfully deleted
I20260806 01:06:57.068509 25291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:57.044276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.22.63.126:40477)
0806 01:06:57.044832 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:57.044840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:57.051610 (+ 6770us) server_negotiation.cc:407] Connection header received
0806 01:06:57.051758 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:57.051764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:57.051862 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:57.051977 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:57.053845 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.055019 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.057069 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.057738 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.057883 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:57.061223 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:57.061232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:57.061237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:57.061282 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:57.065045 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.066026 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.066032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.066035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.066213 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.066651 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.066657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.066660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.067133 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:57.067385 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:57.067842 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 01:06:57.068096 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":233,"threads_started":1}
W20260806 01:06:57.069546 24417 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49574
I20260806 01:06:57.072443 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24527
I20260806 01:06:57.092643 25260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:57.073671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.22.63.65:45361)
0806 01:06:57.074813 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:57.074820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:57.076206 (+ 1386us) server_negotiation.cc:407] Connection header received
0806 01:06:57.076476 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:57.076483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:57.076564 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:57.076636 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:57.078418 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.079565 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.081612 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.082324 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.082453 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:57.085433 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:57.085440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:57.085445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:57.085486 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:57.089348 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.090249 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.090255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.090258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.090423 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.090869 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.090874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.090878 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.091243 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:57.091555 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:57.091968 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 01:06:57.092212 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973}
W20260806 01:06:57.093287 24713 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43432
W20260806 01:06:57.093935 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24527: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:57.094137 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24527
I20260806 01:06:57.125787 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24726
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:06:57.144748 25295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:57.126653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.22.63.66:39411)
0806 01:06:57.128250 (+ 1597us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:57.128257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:57.128279 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:06:57.128361 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:57.128367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:57.128463 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:57.128586 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:57.129765 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.130936 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.132421 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.133099 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.133229 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:57.137366 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:57.137387 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:57.137408 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:57.137483 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:57.141514 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.142278 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.142288 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.142291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.142477 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.142838 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.142843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.142846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.143330 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:57.143642 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:57.143850 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 01:06:57.144106 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1440,"thread_start_us":1299,"threads_started":1}
W20260806 01:06:57.145576 24925 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54026
W20260806 01:06:57.146306 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24726: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:57.146492 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24726
I20260806 01:06:57.172075 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24936
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[24360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:06:57.191284 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:57.172921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40098 (local address 127.22.63.67:43505)
0806 01:06:57.173121 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:57.173125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:57.173146 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:06:57.173343 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:57.173347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:57.173422 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:57.173518 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:57.175214 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.176098 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.177943 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:57.178434 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:57.178543 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:57.183947 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:57.183969 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:57.184006 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:57.184063 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:57.188246 (+ 4183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.188936 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.188945 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.188949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.189132 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:57.189503 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:57.189508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:57.189511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:57.190045 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:57.190392 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:57.190574 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 01:06:57.190850 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 01:06:57.192191 25152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40098
W20260806 01:06:57.192929 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24936: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:06:57.193118 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24936
I20260806 01:06:57.226272 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24378
I20260806 01:06:57.227413 24483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:06:57.345130 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24378
2026-08-06T01:06:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5421 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[25302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[25302](info): set up 2 sockets
Aug 06 01:06:57 dist-test-slave-7f01 krb5kdc[25302](info): commencing operation
krb5kdc: starting...
W20260806 01:06:58.661731 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:06:58 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:06:58.893703 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40009
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:59.237602 25320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:59.237989 25320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:59.238169 25320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:59.245532 25320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:06:59.245685 25320 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:06:59.245790 25320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:59.245908 25320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:59.246007 25320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:06:59.246104 25320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:06:59.256174 25320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40009
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:59.257584 25320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:59.258992 25320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:59.272190 25335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:59.272118 25330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:59.272156 25331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:59.273048 25320 server_base.cc:1034] running on GCE node
I20260806 01:06:59.273927 25320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:59.275493 25320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:59.276773 25320 hybrid_clock.cc:648] HybridClock initialized: now 1785978419276691 us; error 54 us; skew 500 ppm
I20260806 01:06:59.277181 25320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:59 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978419, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:59.280615 25320 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:06:59.282238 25320 webserver.cc:460] Webserver started at http://127.22.63.126:42113/ using document root <none> and password file <none>
I20260806 01:06:59.283008 25320 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:59.283146 25320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:59.283492 25320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:59.286687 25320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "e7df64e2459b42398077e22362efce35"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "8d269aebbc7539b5ee16a09abacdc6c5"
server_key_iv: "8a0af5a2b65b52a5661183d942dd7480"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.287535 25320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "e7df64e2459b42398077e22362efce35"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "8d269aebbc7539b5ee16a09abacdc6c5"
server_key_iv: "8a0af5a2b65b52a5661183d942dd7480"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.293067 25320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 01:06:59.296680 25348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:59.298034 25320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:06:59.298233 25320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "e7df64e2459b42398077e22362efce35"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "8d269aebbc7539b5ee16a09abacdc6c5"
server_key_iv: "8a0af5a2b65b52a5661183d942dd7480"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.298468 25320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:59.313175 25320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:59.317606 25320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:59.317909 25320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:59.340848 25320 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40009
I20260806 01:06:59.340866 25442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40009 every 8 connection(s)
I20260806 01:06:59.342384 25320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:06:59.346387 25443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:06:59.347358 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25320
I20260806 01:06:59.347606 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:06:59.348536 22781 external_mini_cluster.cc:1428] Setting key a70cb0c1965f139fc43c8ab090e7ecef
I20260806 01:06:59.361768 25443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Bootstrap starting.
Aug 06 01:06:59 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:59.365831 25443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Neither blocks nor log segments found. Creating new log.
I20260806 01:06:59.367178 25443 log.cc:824] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Log is configured to *not* fsync() on all Append() calls
I20260806 01:06:59.370409 25443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: No bootstrap required, opened a new log
I20260806 01:06:59.374441 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:59.350888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52338 (local address 127.22.63.126:40009)
0806 01:06:59.351872 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:59.351884 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:59.351936 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:06:59.353620 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:59.353645 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:59.354086 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:59.354434 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:59.356329 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.357693 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.359564 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.360393 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.360533 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:59.365615 (+ 5082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:59.365626 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:59.365642 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:59.365686 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:59.369227 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:59.370009 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:59.370022 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:59.370029 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:59.370216 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:59.370659 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:59.370664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:59.370667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:59.371496 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:59.371801 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:59.372387 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 01:06:59.372908 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":437,"threads_started":1}
I20260806 01:06:59.379678 25443 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } }
I20260806 01:06:59.380121 25443 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:06:59.380208 25443 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7df64e2459b42398077e22362efce35, State: Initialized, Role: FOLLOWER
I20260806 01:06:59.380716 25443 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } }
I20260806 01:06:59.380932 25443 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:06:59.381031 25443 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:06:59.381155 25443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:06:59.383303 25443 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } }
I20260806 01:06:59.383877 25443 leader_election.cc:302] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7df64e2459b42398077e22362efce35; no voters:
I20260806 01:06:59.384531 25443 leader_election.cc:288] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:06:59.384761 25449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:06:59.385828 25449 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [term 1 LEADER]: Becoming Leader. State: Replica: e7df64e2459b42398077e22362efce35, State: Running, Role: LEADER
I20260806 01:06:59.386410 25449 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } }
I20260806 01:06:59.386790 25443 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:06:59.389875 25455 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7df64e2459b42398077e22362efce35. Latest consensus state: current_term: 1 leader_uuid: "e7df64e2459b42398077e22362efce35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } } }
I20260806 01:06:59.390391 25455 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:59.389906 25450 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7df64e2459b42398077e22362efce35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7df64e2459b42398077e22362efce35" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40009 } } }
I20260806 01:06:59.390878 25450 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35 [sys.catalog]: This master's current role is: LEADER
I20260806 01:06:59.397126 25466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:06:59.402678 25466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:06:59.411362 25466 catalog_manager.cc:1347] Generated new cluster ID: 7b3e863660aa4f0591a5c7f11f70f7f2
I20260806 01:06:59.411489 25466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:06:59.434939 25466 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:06:59.436506 25466 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:06:59.445624 25466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Generated new TSK 0
I20260806 01:06:59.446529 25466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:06:59.508900 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40009
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:06:59.863495 25488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:06:59.863864 25488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:06:59.864048 25488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:06:59.871592 25488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:06:59.871775 25488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:06:59.871934 25488 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:06:59.872038 25488 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:06:59.872177 25488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:06:59.888044 25488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:06:59.889617 25488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:06:59.891182 25488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:06:59.905377 25494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:59.905306 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:06:59.905334 25501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:06:59.906533 25488 server_base.cc:1034] running on GCE node
I20260806 01:06:59.907222 25488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:06:59.908255 25488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:06:59.909518 25488 hybrid_clock.cc:648] HybridClock initialized: now 1785978419909474 us; error 54 us; skew 500 ppm
I20260806 01:06:59.909873 25488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:06:59 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978419, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:06:59.913341 25488 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:06:59.915023 25488 webserver.cc:460] Webserver started at http://127.22.63.65:42471/ using document root <none> and password file <none>
I20260806 01:06:59.915832 25488 fs_manager.cc:359] Metadata directory not provided
I20260806 01:06:59.915987 25488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:06:59.916332 25488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:06:59.919392 25488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "119e6bc4fa3042cc8eb0e1110509556f"
server_key_iv: "85b8988bc8436fce324eb1bf15a8494c"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.920235 25488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "119e6bc4fa3042cc8eb0e1110509556f"
server_key_iv: "85b8988bc8436fce324eb1bf15a8494c"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.925590 25488 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:06:59.929296 25510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:59.930668 25488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:06:59.930872 25488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
format_stamp: "Formatted at 2026-08-06 01:06:59 on dist-test-slave-7f01"
server_key: "119e6bc4fa3042cc8eb0e1110509556f"
server_key_iv: "85b8988bc8436fce324eb1bf15a8494c"
server_key_version: "encryptionkey@0"
I20260806 01:06:59.931085 25488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:06:59.948249 25488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:06:59.952937 25488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:06:59.953271 25488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:06:59.956097 25488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:06:59.956234 25488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:06:59.956360 25488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:06:59.956456 25488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:06:59 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978419, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:06:59.984622 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:59.956768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:55441 (local address 127.22.63.126:40009)
0806 01:06:59.957352 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:59.957362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:59.958583 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 01:06:59.960756 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:59.960764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:59.960870 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:59.960995 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:59.963258 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.964612 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.967936 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.968706 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.968892 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:59.975193 (+ 6301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:06:59.975202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:06:59.975210 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:06:59.975262 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:06:59.978852 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:59.980295 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:59.980304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:59.980310 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:59.980509 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:06:59.981195 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:06:59.981202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:06:59.981206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:06:59.981672 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:06:59.981870 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:06:59.983989 (+ 2119us) server_negotiation.cc:299] Negotiation successful
0806 01:06:59.984250 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":316,"threads_started":1}
I20260806 01:06:59.985364 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:59.957544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.65:55441)
0806 01:06:59.958405 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:59.958453 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:59.960454 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:59.961232 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:59.961240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:59.961957 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:59.963070 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:59.963095 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.964832 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.964841 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:59.967694 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:59.967717 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.969032 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.969040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:59.969420 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:06:59.970500 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:06:59.970543 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:06:59.974860 (+ 4317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:06:59.979035 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:59.979046 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:59.979069 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:59.980096 (+ 1027us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:59.980665 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:06:59.980673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:06:59.980677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:06:59.981028 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:06:59.981840 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:06:59.981852 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:06:59.982456 (+ 604us) client_negotiation.cc:769] Sending connection context
0806 01:06:59.982872 (+ 416us) client_negotiation.cc:240] Negotiation successful
0806 01:06:59.983608 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":359,"threads_started":1}
I20260806 01:07:00.000447 25488 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:46501
I20260806 01:07:00.000483 25682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:46501 every 8 connection(s)
I20260806 01:07:00.002032 25488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:00.003847 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25488
I20260806 01:07:00.004429 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:00.005340 22781 external_mini_cluster.cc:1428] Setting key 3bb441eed01a68e6a49acb3b2f237f45
I20260806 01:07:00.005515 25683 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40009
I20260806 01:07:00.006521 25683 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:00.007460 25683 heartbeater.cc:511] Master 127.22.63.126:40009 requested a full tablet report, sending...
I20260806 01:07:00.009879 25368 ts_manager.cc:194] Registered new tserver with Master: 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501)
I20260806 01:07:00.010241 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:59.993092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:55353 (local address 127.22.63.126:40009)
0806 01:06:59.993299 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:06:59.993306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:06:59.994589 (+ 1283us) server_negotiation.cc:407] Connection header received
0806 01:06:59.994811 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:06:59.994816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:06:59.994897 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:06:59.995001 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:06:59.996588 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.997621 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.999445 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.000043 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.000191 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.002274 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:00.002281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:00.002286 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:00.002326 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:00.005828 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.007634 (+ 1806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.007639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.007642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.007908 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.008367 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.008374 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.008378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.008778 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:00.008951 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:00.009575 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 01:07:00.009836 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:07:00.010453 25661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:06:59.993607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.65:55353)
0806 01:06:59.994458 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0806 01:06:59.994491 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:06:59.994677 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:06:59.995045 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:06:59.995050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:06:59.995794 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:06:59.996356 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:59.996376 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:06:59.997802 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:06:59.997806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:06:59.999201 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:06:59.999215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.000263 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.000270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:00.000473 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.001216 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:00.001235 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:00.002038 (+ 803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:00.005993 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.006000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.006003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.007519 (+ 1516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.008042 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.008050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.008053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.008214 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.009036 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:00.009044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:00.009270 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 01:07:00.009443 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 01:07:00.009846 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":491,"threads_started":1}
I20260806 01:07:00.011708 25368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:55441
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:07:00.059008 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40009
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:00.422768 25687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:00.423084 25687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:00.423264 25687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:00.430732 25687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:00.430848 25687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:00.431005 25687 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:07:00.431087 25687 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:07:00.431228 25687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:00.441519 25687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:00.442934 25687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:00.444363 25687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:00.458005 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:00.458037 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:00.458170 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:00.459671 25687 server_base.cc:1034] running on GCE node
I20260806 01:07:00.460430 25687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:00.461433 25687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:00.462648 25687 hybrid_clock.cc:648] HybridClock initialized: now 1785978420462613 us; error 35 us; skew 500 ppm
I20260806 01:07:00.462937 25687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:00 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978420, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:00.466351 25687 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:00.468101 25687 webserver.cc:460] Webserver started at http://127.22.63.66:41603/ using document root <none> and password file <none>
I20260806 01:07:00.468837 25687 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:00.468990 25687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:00.469323 25687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:00.472395 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "d55c9fca009740e998167d129795be0c"
format_stamp: "Formatted at 2026-08-06 01:07:00 on dist-test-slave-7f01"
server_key: "efa9b675353f4d5bd5dab3ff86ab2051"
server_key_iv: "ae74a0638bffd0bf93cc683ead352446"
server_key_version: "encryptionkey@0"
I20260806 01:07:00.473176 25687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "d55c9fca009740e998167d129795be0c"
format_stamp: "Formatted at 2026-08-06 01:07:00 on dist-test-slave-7f01"
server_key: "efa9b675353f4d5bd5dab3ff86ab2051"
server_key_iv: "ae74a0638bffd0bf93cc683ead352446"
server_key_version: "encryptionkey@0"
I20260806 01:07:00.478562 25687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:07:00.482195 25706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:00.483544 25687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:07:00.483800 25687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "d55c9fca009740e998167d129795be0c"
format_stamp: "Formatted at 2026-08-06 01:07:00 on dist-test-slave-7f01"
server_key: "efa9b675353f4d5bd5dab3ff86ab2051"
server_key_iv: "ae74a0638bffd0bf93cc683ead352446"
server_key_version: "encryptionkey@0"
I20260806 01:07:00.484046 25687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:00.503805 25687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:00.508364 25687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:00.508680 25687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:00.512030 25687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:00.512161 25687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:00.512287 25687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:00.512456 25687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 06 01:07:00 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978420, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:00.537340 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:00.512154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:45209 (local address 127.22.63.126:40009)
0806 01:07:00.512692 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:00.512699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:00.514016 (+ 1317us) server_negotiation.cc:407] Connection header received
0806 01:07:00.516250 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:00.516256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:00.516366 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:00.516491 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:00.518785 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.519948 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.522583 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.523313 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.523493 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.529420 (+ 5927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:00.529427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:00.529430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:00.529512 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:00.533472 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.534346 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.534352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.534356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.534512 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.535071 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.535077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.535080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.535387 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:00.535578 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:00.536649 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0806 01:07:00.536902 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":269,"threads_started":1}
I20260806 01:07:00.538455 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:00.512868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.66:45209)
0806 01:07:00.513802 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:00.513863 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:00.515941 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:00.516748 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:00.516760 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:00.517565 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:00.518557 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:00.518591 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.520134 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.520141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:00.522386 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:00.522413 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.523944 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.523965 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:00.524259 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.525210 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:00.525237 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:00.529077 (+ 3840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:00.533677 (+ 4600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.533689 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.533717 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.534195 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.534634 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.534639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.534644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.534923 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.535531 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:00.535541 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:00.536144 (+ 603us) client_negotiation.cc:769] Sending connection context
0806 01:07:00.536570 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 01:07:00.537191 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":388,"threads_started":1}
I20260806 01:07:00.561478 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:00.545231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59433 (local address 127.22.63.126:40009)
0806 01:07:00.545427 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:00.545431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:00.546566 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 01:07:00.546848 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:00.546853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:00.546931 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:00.547045 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:00.548360 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.549477 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.550865 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.551292 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.551451 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.554150 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:00.554180 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:00.554186 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:00.554235 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:00.557737 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.558907 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.558911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.558913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.559043 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:00.559592 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:00.559599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:00.559603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:00.560196 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:00.560381 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:00.560845 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 01:07:00.561100 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:07:00.561617 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:00.545369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.66:59433)
0806 01:07:00.546431 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:00.546455 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:00.546675 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:00.547077 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:00.547085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:00.547597 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:00.548119 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:00.548137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.549636 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.549641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:00.550704 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:00.550716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:00.551498 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:00.551507 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:00.551683 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:00.552495 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:00.552531 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:00.553855 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:00.557971 (+ 4116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.557979 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.557985 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.558692 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.559223 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:00.559230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:00.559236 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:00.559456 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:00.560354 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:00.560359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:00.560563 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 01:07:00.560732 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 01:07:00.561116 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":731,"threads_started":1}
I20260806 01:07:00.568300 25687 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:36249
I20260806 01:07:00.568353 25892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:36249 every 8 connection(s)
I20260806 01:07:00.569599 25687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:00.572094 25893 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40009
I20260806 01:07:00.572372 25893 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:00.573105 25893 heartbeater.cc:511] Master 127.22.63.126:40009 requested a full tablet report, sending...
I20260806 01:07:00.574642 25368 ts_manager.cc:194] Registered new tserver with Master: d55c9fca009740e998167d129795be0c (127.22.63.66:36249)
I20260806 01:07:00.575480 25368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:45209
I20260806 01:07:00.577509 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25687
I20260806 01:07:00.577709 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:00.578259 22781 external_mini_cluster.cc:1428] Setting key c5839c5f1f156771fff099d5ac810a7b
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:07:00.632371 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40009
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:00.980049 25897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:00.980396 25897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:00.980580 25897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:00.988540 25897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:00.988682 25897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:00.988838 25897 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 01:07:00.988931 25897 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:07:00.989048 25897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:00.999970 25897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:01.001410 25897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:01.002858 25897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 01:07:01.015236 25683 heartbeater.cc:503] Master 127.22.63.126:40009 was elected leader, sending a full tablet report...
W20260806 01:07:01.017268 25906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:01.017469 25903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:01.017508 25908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:01.019105 25897 server_base.cc:1034] running on GCE node
I20260806 01:07:01.019783 25897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:01.020758 25897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:01.021965 25897 hybrid_clock.cc:648] HybridClock initialized: now 1785978421021919 us; error 38 us; skew 500 ppm
I20260806 01:07:01.022250 25897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:01 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978421, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:01.030159 25897 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:01.031653 25897 webserver.cc:460] Webserver started at http://127.22.63.67:37813/ using document root <none> and password file <none>
I20260806 01:07:01.032423 25897 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:01.032531 25897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:01.032796 25897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:01.036003 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "aed9a11b02b64a128dab88810f826f7c"
format_stamp: "Formatted at 2026-08-06 01:07:01 on dist-test-slave-7f01"
server_key: "def484a75d1012a6712328db013b3146"
server_key_iv: "cf66ddfd7b6c1e6a9f068ea268b8cec5"
server_key_version: "encryptionkey@0"
I20260806 01:07:01.036823 25897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "aed9a11b02b64a128dab88810f826f7c"
format_stamp: "Formatted at 2026-08-06 01:07:01 on dist-test-slave-7f01"
server_key: "def484a75d1012a6712328db013b3146"
server_key_iv: "cf66ddfd7b6c1e6a9f068ea268b8cec5"
server_key_version: "encryptionkey@0"
I20260806 01:07:01.042608 25897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:01.046226 25916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:01.047581 25897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:07:01.047719 25897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "aed9a11b02b64a128dab88810f826f7c"
format_stamp: "Formatted at 2026-08-06 01:07:01 on dist-test-slave-7f01"
server_key: "def484a75d1012a6712328db013b3146"
server_key_iv: "cf66ddfd7b6c1e6a9f068ea268b8cec5"
server_key_version: "encryptionkey@0"
I20260806 01:07:01.047960 25897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:01.068025 25897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:01.072419 25897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:01.072680 25897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:01.075284 25897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:01.075438 25897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:01.075574 25897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:01.075670 25897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:07:01 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978421, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:01.102227 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.076169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:34103 (local address 127.22.63.126:40009)
0806 01:07:01.080142 (+ 3973us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.080149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.080174 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:07:01.080557 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.080562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.080654 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.080788 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:01.082622 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.083817 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.086627 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.087420 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.087604 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.093480 (+ 5876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:01.093491 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:01.093494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:01.093529 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:01.097256 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.098409 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.098418 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.098422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.098610 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.099157 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.099165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.099169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.099452 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:01.099619 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.101760 (+ 2141us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.101929 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3840,"thread_start_us":3706,"threads_started":1}
I20260806 01:07:01.103276 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.076758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.67:34103)
0806 01:07:01.077648 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.077694 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.080234 (+ 2540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.081047 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.081058 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.081578 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:01.082413 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.082445 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.084072 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.084081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.086398 (+ 2317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.086434 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.088010 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.088027 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.088313 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.089257 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:01.089304 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:01.093221 (+ 3917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:01.097450 (+ 4229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.097465 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.097485 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.098195 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.098759 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.098765 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.098779 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.099010 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.099595 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:01.099603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:01.100077 (+ 474us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.100509 (+ 432us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.100949 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":398,"threads_started":1}
I20260806 01:07:01.125471 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.110585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:47427 (local address 127.22.63.126:40009)
0806 01:07:01.110749 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.110753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.111442 (+ 689us) server_negotiation.cc:407] Connection header received
0806 01:07:01.111646 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.111650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.111711 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.111848 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:01.113566 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.114548 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.116444 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.117081 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.117236 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.119111 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:01.119117 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:01.119124 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:01.119160 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:01.122479 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.123364 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.123369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.123372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.123515 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.123967 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.123973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.123976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.124265 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:01.124437 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.124836 (+ 399us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.125075 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:07:01.125574 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.110687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.22.63.67:47427)
0806 01:07:01.111330 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.111349 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.111512 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.112065 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.112072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.112758 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:01.113331 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.113350 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.114778 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.114781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.116205 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.116218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.117318 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.117326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.117496 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.118162 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:01.118180 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:01.118913 (+ 733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:01.122612 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.122618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.122622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.123191 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.123638 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.123643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.123652 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.123850 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.124391 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:01.124396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:01.124587 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.124739 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.125091 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":376,"threads_started":1}
I20260806 01:07:01.130192 25897 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:41069
I20260806 01:07:01.130239 26107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:41069 every 8 connection(s)
I20260806 01:07:01.131431 25897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:01.133599 26108 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40009
I20260806 01:07:01.133867 26108 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:01.134523 26108 heartbeater.cc:511] Master 127.22.63.126:40009 requested a full tablet report, sending...
I20260806 01:07:01.136152 25368 ts_manager.cc:194] Registered new tserver with Master: aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:01.136926 25368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:34103
I20260806 01:07:01.137529 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25897
I20260806 01:07:01.137725 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:01.138186 22781 external_mini_cluster.cc:1428] Setting key f4deae8d773a388c5b0902f12b111b6c
I20260806 01:07:01.141235 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:07:01 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:07:01 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:01.196161 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.181841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.22.63.126:40009)
0806 01:07:01.182031 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.182036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.182491 (+ 455us) server_negotiation.cc:407] Connection header received
0806 01:07:01.182588 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.182593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.182690 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.182794 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:01.184304 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.185092 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.186322 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.186864 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.186959 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.191268 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:01.191273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:01.191276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:01.191310 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:01.193713 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.194326 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.194330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.194333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.194473 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.194828 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.194832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.194834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.195127 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:01.195339 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.195556 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.195714 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:07:01.200487 25368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:01.207828 25368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:01.232915 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.220169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.22.63.67:41069)
0806 01:07:01.222195 (+ 2026us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.222201 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.222225 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:07:01.222311 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.222317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.222591 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.222799 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.224077 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.226984 (+ 2907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.230014 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.231547 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.231693 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.231979 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.232348 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.232546 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1902,"thread_start_us":1746,"threads_started":1}
I20260806 01:07:01.233210 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.220686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.22.63.66:36249)
0806 01:07:01.221772 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.221779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.222647 (+ 868us) server_negotiation.cc:407] Connection header received
0806 01:07:01.222906 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.222914 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.223142 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.223326 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.224548 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.226563 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.230366 (+ 3803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.231725 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.231857 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.232050 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.232700 (+ 650us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.232886 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":784,"threads_started":1}
I20260806 01:07:01.233747 26127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.219965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:41069 (local address 127.0.0.1:41174)
0806 01:07:01.220687 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.220730 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.220897 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.223213 (+ 2316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.223225 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.223266 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.223817 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.223833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.227214 (+ 3381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.227223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.229831 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.229846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.231809 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.231816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.232036 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.232097 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.232234 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.233326 (+ 1092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":1,"thread_start_us":334,"threads_started":1}
I20260806 01:07:01.234163 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.220151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36249 (local address 127.0.0.1:46560)
0806 01:07:01.222527 (+ 2376us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.222546 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.222687 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.223649 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.223656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.223697 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.224310 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.224328 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.226825 (+ 2497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.226834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.230161 (+ 3327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.230177 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.232213 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.232219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.232444 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.232486 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.233502 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.233674 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2228,"mutex_wait_us":4,"thread_start_us":2255,"threads_started":1}
I20260806 01:07:01.237567 25777 tablet_service.cc:1467] Processing CreateTablet for tablet 444d5a48a47d4ea4956c231a94724917 (DEFAULT_TABLE table=test-table [id=1908e1425785415291385920595147ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.237610 25999 tablet_service.cc:1467] Processing CreateTablet for tablet 444d5a48a47d4ea4956c231a94724917 (DEFAULT_TABLE table=test-table [id=1908e1425785415291385920595147ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.239117 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.222671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46501 (local address 127.0.0.1:40128)
0806 01:07:01.229333 (+ 6662us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.229351 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.229487 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.230110 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.230115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.230147 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.230515 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.230524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.232139 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.232145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.235080 (+ 2935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.235845 (+ 765us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.238337 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.238346 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.238539 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.238572 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.238676 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.238810 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6528,"thread_start_us":6356,"threads_started":1}
I20260806 01:07:01.239166 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d5a48a47d4ea4956c231a94724917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.239384 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.223078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.22.63.65:46501)
0806 01:07:01.228957 (+ 5879us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.228967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.229480 (+ 513us) server_negotiation.cc:407] Connection header received
0806 01:07:01.229630 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.229637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.229891 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.230132 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.230686 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.231660 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.236042 (+ 4382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.238109 (+ 2067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.238274 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.238479 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.238739 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.238927 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5702,"thread_start_us":5557,"threads_started":1}
I20260806 01:07:01.239426 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d5a48a47d4ea4956c231a94724917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.242903 25577 tablet_service.cc:1467] Processing CreateTablet for tablet 444d5a48a47d4ea4956c231a94724917 (DEFAULT_TABLE table=test-table [id=1908e1425785415291385920595147ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.244305 25577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 444d5a48a47d4ea4956c231a94724917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.252408 26137 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Bootstrap starting.
I20260806 01:07:01.254240 26138 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Bootstrap starting.
I20260806 01:07:01.256400 26137 tablet_bootstrap.cc:654] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.257689 26138 tablet_bootstrap.cc:654] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.257755 26137 log.cc:824] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:01.258257 26139 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Bootstrap starting.
I20260806 01:07:01.258984 26138 log.cc:824] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:01.261976 26137 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: No bootstrap required, opened a new log
I20260806 01:07:01.261955 26139 tablet_bootstrap.cc:654] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.262221 26138 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: No bootstrap required, opened a new log
I20260806 01:07:01.262301 26137 ts_tablet_manager.cc:1397] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 01:07:01.262521 26138 ts_tablet_manager.cc:1397] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260806 01:07:01.263270 26139 log.cc:824] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:01.266330 26139 tablet_bootstrap.cc:492] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: No bootstrap required, opened a new log
I20260806 01:07:01.266592 26139 ts_tablet_manager.cc:1397] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260806 01:07:01.270637 26137 raft_consensus.cc:348] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.271114 26137 raft_consensus.cc:374] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.271221 26137 raft_consensus.cc:729] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9a11b02b64a128dab88810f826f7c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.271986 26137 consensus_queue.cc:260] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.273137 26108 heartbeater.cc:503] Master 127.22.63.126:40009 was elected leader, sending a full tablet report...
I20260806 01:07:01.272894 26138 raft_consensus.cc:348] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.273303 26138 raft_consensus.cc:374] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.273397 26138 raft_consensus.cc:729] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55c9fca009740e998167d129795be0c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.274055 26137 ts_tablet_manager.cc:1428] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260806 01:07:01.273985 26138 consensus_queue.cc:260] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.275125 25893 heartbeater.cc:503] Master 127.22.63.126:40009 was elected leader, sending a full tablet report...
I20260806 01:07:01.275754 26138 ts_tablet_manager.cc:1428] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:07:01.277114 26139 raft_consensus.cc:348] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.277596 26139 raft_consensus.cc:374] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.277707 26139 raft_consensus.cc:729] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.278338 26139 consensus_queue.cc:260] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.280789 26150 raft_consensus.cc:482] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:01.281159 26150 raft_consensus.cc:504] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.282075 26139 ts_tablet_manager.cc:1428] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260806 01:07:01.283100 26150 leader_election.cc:288] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069), d55c9fca009740e998167d129795be0c (127.22.63.66:36249)
I20260806 01:07:01.291128 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.283363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36249 (local address 127.22.63.65:34999)
0806 01:07:01.283977 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.283999 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.284149 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.284511 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.284515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.284540 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.284987 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.285004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.286442 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.286448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.288863 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.288877 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.290391 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.290398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.290590 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.290681 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.290767 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.290867 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":821,"threads_started":1}
I20260806 01:07:01.291235 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.283928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:34999 (local address 127.22.63.66:36249)
0806 01:07:01.284140 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.284146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.284168 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:07:01.284327 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.284333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.284415 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.284505 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.285174 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.286271 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.289111 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.290160 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.290330 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.290390 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.290827 (+ 437us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.290972 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:07:01.292706 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.283373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:41069 (local address 127.22.63.65:45815)
0806 01:07:01.285816 (+ 2443us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.285837 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.286024 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.286548 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.286556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.286592 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.287244 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.287265 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.288778 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.288789 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.290553 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.290567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.292169 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.292174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.292287 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.292311 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.292386 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.292475 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2303,"thread_start_us":2162,"threads_started":1}
I20260806 01:07:01.292583 25822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444d5a48a47d4ea4956c231a94724917" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55c9fca009740e998167d129795be0c" is_pre_election: true
I20260806 01:07:01.292820 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.283910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:45815 (local address 127.22.63.67:41069)
0806 01:07:01.284147 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.284155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.285915 (+ 1760us) server_negotiation.cc:407] Connection header received
0806 01:07:01.286166 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.286173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.286253 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.286319 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.287452 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.288611 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.290747 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.291951 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.292167 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.292249 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.292439 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.292573 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:07:01.293215 25822 raft_consensus.cc:2454] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 0.
I20260806 01:07:01.293808 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444d5a48a47d4ea4956c231a94724917" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c" is_pre_election: true
I20260806 01:07:01.293926 25513 leader_election.cc:302] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.294332 26035 raft_consensus.cc:2454] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 0.
I20260806 01:07:01.294364 26150 raft_consensus.cc:2781] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:01.294517 26150 raft_consensus.cc:482] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:01.294634 26150 raft_consensus.cc:3037] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.296998 26150 raft_consensus.cc:504] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.297636 26150 leader_election.cc:288] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 election: Requested vote from peers aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069), d55c9fca009740e998167d129795be0c (127.22.63.66:36249)
I20260806 01:07:01.298133 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444d5a48a47d4ea4956c231a94724917" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c"
I20260806 01:07:01.298166 25822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "444d5a48a47d4ea4956c231a94724917" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55c9fca009740e998167d129795be0c"
I20260806 01:07:01.298338 26035 raft_consensus.cc:3037] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.298414 25822 raft_consensus.cc:3037] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.300853 26035 raft_consensus.cc:2454] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 1.
I20260806 01:07:01.300894 25822 raft_consensus.cc:2454] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 1.
I20260806 01:07:01.301311 25513 leader_election.cc:302] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.301651 26150 raft_consensus.cc:2781] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:01.301851 26150 raft_consensus.cc:686] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 LEADER]: Becoming Leader. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Running, Role: LEADER
I20260806 01:07:01.302428 26150 consensus_queue.cc:237] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } }
I20260806 01:07:01.306528 25368 catalog_manager.cc:5495] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65). New cstate: current_term: 1 leader_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } health_report { overall_health: HEALTHY } } }
W20260806 01:07:01.321825 25894 tablet.cc:2369] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 01:07:01 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978419, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:01.369472 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.354811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.22.63.126:40009)
0806 01:07:01.355044 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.355051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.355732 (+ 681us) server_negotiation.cc:407] Connection header received
0806 01:07:01.355936 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.355940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.356020 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.356106 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:01.357553 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.358337 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.359616 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.360176 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.360276 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.363918 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:01.363923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:01.363926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:01.363958 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:01.366545 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.367268 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.367273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.367276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.367425 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:01.367862 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:01.367868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:01.367871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:01.368132 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:01.368292 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.368744 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.369072 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":70}
I20260806 01:07:01.369508 26165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.354914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40009 (local address 127.0.0.1:52350)
0806 01:07:01.355615 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.355634 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.355811 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.356163 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.356168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.357092 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:01.357409 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.357419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.358466 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.358470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.359475 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.359487 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.360382 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.360387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.360521 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.361293 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:01.361316 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:01.363645 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:01.366684 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.366699 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.366708 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.367142 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.367538 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:01.367544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:01.367548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:01.367680 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:01.368278 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:01.368286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:01.368464 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.368646 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.368946 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":360,"threads_started":1}
I20260806 01:07:01.374830 25368 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:52350:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:07:01.376035 25368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260806 01:07:01.383443 26109 tablet.cc:2369] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:01.383286 25777 tablet_service.cc:1467] Processing CreateTablet for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fe4611845c4c018d047122e782c642]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:01.383227 25577 tablet_service.cc:1467] Processing CreateTablet for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fe4611845c4c018d047122e782c642]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:01.384383 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.384444 25577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.384078 25999 tablet_service.cc:1467] Processing CreateTablet for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78fe4611845c4c018d047122e782c642]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:01.384939 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd4cd6a4a69e4d4da0b48644b5c65e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.391640 26139 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: Bootstrap starting.
I20260806 01:07:01.391873 26138 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: Bootstrap starting.
I20260806 01:07:01.392339 26137 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: Bootstrap starting.
I20260806 01:07:01.393576 26139 tablet_bootstrap.cc:654] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.393642 26138 tablet_bootstrap.cc:654] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.394066 26137 tablet_bootstrap.cc:654] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.396700 26138 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: No bootstrap required, opened a new log
I20260806 01:07:01.396881 26138 ts_tablet_manager.cc:1397] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 01:07:01.396930 26139 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: No bootstrap required, opened a new log
I20260806 01:07:01.397100 26139 ts_tablet_manager.cc:1397] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 01:07:01.397445 26137 tablet_bootstrap.cc:492] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: No bootstrap required, opened a new log
I20260806 01:07:01.397586 26137 ts_tablet_manager.cc:1397] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260806 01:07:01.397782 26139 raft_consensus.cc:348] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.397969 26139 raft_consensus.cc:374] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.397782 26138 raft_consensus.cc:348] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.398032 26139 raft_consensus.cc:729] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.398049 26138 raft_consensus.cc:374] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.398149 26138 raft_consensus.cc:729] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55c9fca009740e998167d129795be0c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.398260 26139 consensus_queue.cc:260] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.398397 26138 consensus_queue.cc:260] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.398449 26137 raft_consensus.cc:348] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.398699 26137 raft_consensus.cc:374] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.398799 26137 raft_consensus.cc:729] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9a11b02b64a128dab88810f826f7c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.398726 26150 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.399029 26150 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.398910 26148 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.399052 26137 consensus_queue.cc:260] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.399269 26148 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.399696 26137 ts_tablet_manager.cc:1428] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 01:07:01.399567 26146 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.399976 26146 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.400606 26138 ts_tablet_manager.cc:1428] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 01:07:01.401160 26139 ts_tablet_manager.cc:1428] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260806 01:07:01.446887 26148 raft_consensus.cc:482] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:01.447196 26148 raft_consensus.cc:504] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.449029 26148 leader_election.cc:288] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:01.457212 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.449662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:50083 (local address 127.22.63.67:41069)
0806 01:07:01.449913 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.449922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.450538 (+ 616us) server_negotiation.cc:407] Connection header received
0806 01:07:01.450803 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.450810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.450902 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.451007 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.451931 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.453086 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.455157 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.456134 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.456300 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.456374 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.456750 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.456890 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 01:07:01.457554 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.449310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:41069 (local address 127.22.63.66:50083)
0806 01:07:01.450413 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.450430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.450608 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.451038 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.451043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.451086 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.451637 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.451652 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.453235 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.453241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.454993 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.455007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.456382 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.456389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.456560 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.456600 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.456705 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.456854 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"mutex_wait_us":21,"spinlock_wait_cycles":1792,"thread_start_us":726,"threads_started":1}
I20260806 01:07:01.458333 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.449423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:48567 (local address 127.22.63.65:46501)
0806 01:07:01.449816 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.449831 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.450579 (+ 748us) server_negotiation.cc:407] Connection header received
0806 01:07:01.450786 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.450790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.450909 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.451017 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:01.451653 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.452834 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.455846 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.456952 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.457120 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.457192 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.457951 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.458093 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260806 01:07:01.458431 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.449324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46501 (local address 127.22.63.66:48567)
0806 01:07:01.450455 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.450471 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.450630 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.451032 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.451037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.451075 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:01.451473 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.451483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.452981 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.452989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.455652 (+ 2663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.455669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.457381 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.457390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.457564 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.457589 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.457919 (+ 330us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.458049 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":63,"spinlock_wait_cycles":3456,"thread_start_us":819,"threads_started":1}
I20260806 01:07:01.458941 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4cd6a4a69e4d4da0b48644b5c65e4b" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c" is_pre_election: true
I20260806 01:07:01.459235 26035 raft_consensus.cc:2454] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:01.459474 25607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4cd6a4a69e4d4da0b48644b5c65e4b" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" is_pre_election: true
I20260806 01:07:01.459779 25710 leader_election.cc:302] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed9a11b02b64a128dab88810f826f7c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.459933 25607 raft_consensus.cc:2454] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:01.460307 26148 raft_consensus.cc:2781] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:01.460474 26148 raft_consensus.cc:482] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:01.460594 26148 raft_consensus.cc:3037] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.462810 26148 raft_consensus.cc:504] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.463471 26148 leader_election.cc:288] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Requested vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:01.463994 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4cd6a4a69e4d4da0b48644b5c65e4b" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c"
I20260806 01:07:01.464017 25607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd4cd6a4a69e4d4da0b48644b5c65e4b" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
I20260806 01:07:01.464218 25607 raft_consensus.cc:3037] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.464246 26035 raft_consensus.cc:3037] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.467375 25607 raft_consensus.cc:2454] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:01.467357 26035 raft_consensus.cc:2454] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:01.467844 25710 leader_election.cc:302] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed9a11b02b64a128dab88810f826f7c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.468138 26148 raft_consensus.cc:2781] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:01.468590 26148 raft_consensus.cc:686] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [term 1 LEADER]: Becoming Leader. State: Replica: d55c9fca009740e998167d129795be0c, State: Running, Role: LEADER
I20260806 01:07:01.469082 26148 consensus_queue.cc:237] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.473851 25365 catalog_manager.cc:5495] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c reported cstate change: term changed from 0 to 1, leader changed from <none> to d55c9fca009740e998167d129795be0c (127.22.63.66). New cstate: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:01.472647 26173 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: TxnStatusTablet state changed. Reason: New leader d55c9fca009740e998167d129795be0c. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.474598 26173 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:01.475371 26176 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:07:01.475642 26176 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:07:01.498786 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.492360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.22.63.66:36249)
0806 01:07:01.492513 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.492519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.492562 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:07:01.492910 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.492914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.492992 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.493105 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.493542 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.494328 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.496357 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.496783 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.496905 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.497509 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.498079 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.498194 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.498358 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.498428 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 01:07:01.498857 26165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.492187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36249 (local address 127.0.0.1:46574)
0806 01:07:01.492484 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.492506 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.492661 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.493169 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.493174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.493189 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:01.493400 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.493412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.494468 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.494474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.496165 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.496182 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.496979 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.496988 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.497220 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.497297 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.498204 (+ 907us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.498228 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.498326 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.498503 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:07:01.507627 25607 raft_consensus.cc:1261] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:01.507654 26035 raft_consensus.cc:1261] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 01:07:01.507812 25684 tablet.cc:2369] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:01.508879 26173 consensus_queue.cc:1035] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:01.509234 26148 consensus_queue.cc:1035] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:01.520572 26146 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: TxnStatusTablet state changed. Reason: New leader d55c9fca009740e998167d129795be0c. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.520969 26146 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.523070 26180 mvcc.cc:205] Tried to move back new op lower bound from 7315367614485245952 to 7315367614341451776. Current Snapshot: MvccSnapshot[applied={T|T < 7315367614485245952}]
I20260806 01:07:01.523314 26150 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: TxnStatusTablet state changed. Reason: New leader d55c9fca009740e998167d129795be0c. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.523627 26150 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.523720 26186 mvcc.cc:205] Tried to move back new op lower bound from 7315367614485245952 to 7315367614341451776. Current Snapshot: MvccSnapshot[applied={T|T < 7315367614485245952}]
I20260806 01:07:01.524143 26173 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.524407 26173 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:01.526682 26148 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.526993 26148 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P d55c9fca009740e998167d129795be0c: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:01.529609 26146 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.529887 26146 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P aed9a11b02b64a128dab88810f826f7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.535683 26150 tablet_replica.cc:439] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } } }
I20260806 01:07:01.536055 26150 tablet_replica.cc:442] T fd4cd6a4a69e4d4da0b48644b5c65e4b P 4f98199ffebb4f9a9ef39f44f1f5984c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:01.543848 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.535967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.22.63.65:46501)
0806 01:07:01.536147 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.536153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.537223 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 01:07:01.537331 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.537337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.537442 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.537612 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.538104 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.539236 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.541315 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.541890 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.541995 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.542575 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.543123 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.543243 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.543403 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.543476 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:07:01.562778 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.556208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:55507 (local address 127.22.63.66:36249)
0806 01:07:01.556453 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.556460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.556805 (+ 345us) server_negotiation.cc:407] Connection header received
0806 01:07:01.557034 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.557038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.557107 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.557187 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.557817 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.558613 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.560734 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.561350 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.561462 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.562105 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.562225 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.562299 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.562483 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.562555 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:07:01.563021 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.556010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36249 (local address 127.22.63.65:55507)
0806 01:07:01.556691 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.556713 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.556872 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.557235 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.557240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.557253 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:01.557633 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.557644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.558747 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.558754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.560564 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.560581 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.561580 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.561590 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.561770 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.561852 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.562334 (+ 482us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.562346 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.562436 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.562602 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":295,"threads_started":1}
I20260806 01:07:01.575207 25822 raft_consensus.cc:1261] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Refusing update from remote peer 4f98199ffebb4f9a9ef39f44f1f5984c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:01.575182 26035 raft_consensus.cc:1261] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Refusing update from remote peer 4f98199ffebb4f9a9ef39f44f1f5984c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:01.575960 26149 consensus_queue.cc:1035] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:01.576434 26150 consensus_queue.cc:1035] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:01.598802 26183 mvcc.cc:205] Tried to move back new op lower bound from 7315367614764285952 to 7315367613658656768. Current Snapshot: MvccSnapshot[applied={T|T < 7315367614764285952}]
I20260806 01:07:01.642802 26136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.634891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:33143 (local address 127.22.63.65:46501)
0806 01:07:01.635105 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.635111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.635924 (+ 813us) server_negotiation.cc:407] Connection header received
0806 01:07:01.636289 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.636295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.636384 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.636520 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.637333 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.638423 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.640199 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.640983 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.641133 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.641952 (+ 819us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.642109 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.642200 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.642384 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.642483 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:07:01.643014 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.634957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46501 (local address 127.22.63.66:33143)
0806 01:07:01.635731 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.635823 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.636090 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.636580 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.636587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.636602 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:01.637098 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.637117 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.638619 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.638625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.640011 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.640031 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.641439 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.641449 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.641683 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.641766 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.642228 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.642242 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.642342 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.642528 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":432,"threads_started":1}
I20260806 01:07:01.689602 25366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:01.689953 25366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:01.695051 25366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Sending DeleteTablet for 3 replicas of tablet 444d5a48a47d4ea4956c231a94724917
I20260806 01:07:01.696463 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet 444d5a48a47d4ea4956c231a94724917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:01 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:41174
I20260806 01:07:01.696522 25777 tablet_service.cc:1514] Processing DeleteTablet for tablet 444d5a48a47d4ea4956c231a94724917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:01 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46560
I20260806 01:07:01.697960 25577 tablet_service.cc:1514] Processing DeleteTablet for tablet 444d5a48a47d4ea4956c231a94724917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:01 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40128
I20260806 01:07:01.706037 25366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 01:07:01.706542 26216 tablet_replica.cc:330] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: stopping tablet replica
W20260806 01:07:01.706993 25366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:01.707291 26216 raft_consensus.cc:2229] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:01.707794 26216 raft_consensus.cc:2258] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:01.715210 25575 tablet_service.cc:1467] Processing CreateTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa (DEFAULT_TABLE table=test-table [id=dd9802ccaf424e218ec247678408d52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.715960 25575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92c726fb1814ad88d2b18ce5a077dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.720403 25776 tablet_service.cc:1467] Processing CreateTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa (DEFAULT_TABLE table=test-table [id=dd9802ccaf424e218ec247678408d52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.721134 25776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92c726fb1814ad88d2b18ce5a077dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.721762 25997 tablet_service.cc:1467] Processing CreateTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa (DEFAULT_TABLE table=test-table [id=dd9802ccaf424e218ec247678408d52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:01.722262 25997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a92c726fb1814ad88d2b18ce5a077dfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:01.727990 26216 ts_tablet_manager.cc:1905] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:01.728896 26139 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Bootstrap starting.
I20260806 01:07:01.729547 26137 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Bootstrap starting.
I20260806 01:07:01.730852 26139 tablet_bootstrap.cc:654] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.731294 26137 tablet_bootstrap.cc:654] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.732059 26138 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Bootstrap starting.
I20260806 01:07:01.751814 26138 tablet_bootstrap.cc:654] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:01.753521 26137 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: No bootstrap required, opened a new log
I20260806 01:07:01.753746 26137 ts_tablet_manager.cc:1397] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Time spent bootstrapping tablet: real 0.024s user 0.019s sys 0.004s
I20260806 01:07:01.754710 26137 raft_consensus.cc:348] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.755082 26137 raft_consensus.cc:374] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.755163 26137 raft_consensus.cc:729] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9a11b02b64a128dab88810f826f7c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.755461 26137 consensus_queue.cc:260] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.756744 26139 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: No bootstrap required, opened a new log
I20260806 01:07:01.756951 26139 ts_tablet_manager.cc:1397] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.008s
I20260806 01:07:01.757757 26139 raft_consensus.cc:348] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.758030 26139 raft_consensus.cc:374] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.758116 26139 raft_consensus.cc:729] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.759203 26137 ts_tablet_manager.cc:1428] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 01:07:01.759518 26217 tablet_replica.cc:330] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: stopping tablet replica
I20260806 01:07:01.760030 26217 raft_consensus.cc:2229] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:01.760816 26217 raft_consensus.cc:2258] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:01.761642 26213 tablet_replica.cc:330] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: stopping tablet replica
I20260806 01:07:01.762118 26213 raft_consensus.cc:2229] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:01.762478 26213 raft_consensus.cc:2258] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:01.762967 26139 consensus_queue.cc:260] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.763891 26139 ts_tablet_manager.cc:1428] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent starting tablet: real 0.007s user 0.002s sys 0.001s
I20260806 01:07:01.766089 26213 ts_tablet_manager.cc:1905] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:01.770931 26217 ts_tablet_manager.cc:1905] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:01.772001 26138 tablet_bootstrap.cc:492] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: No bootstrap required, opened a new log
I20260806 01:07:01.772274 26138 ts_tablet_manager.cc:1397] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.007s
I20260806 01:07:01.773264 26138 raft_consensus.cc:348] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.773630 26138 raft_consensus.cc:374] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:01.773775 26138 raft_consensus.cc:729] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55c9fca009740e998167d129795be0c, State: Initialized, Role: FOLLOWER
I20260806 01:07:01.774163 26138 consensus_queue.cc:260] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.774536 26213 ts_tablet_manager.cc:1918] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:01.774724 26213 log.cc:1196] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/444d5a48a47d4ea4956c231a94724917
I20260806 01:07:01.775199 26138 ts_tablet_manager.cc:1428] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:01.775337 26213 ts_tablet_manager.cc:1939] T 444d5a48a47d4ea4956c231a94724917 P aed9a11b02b64a128dab88810f826f7c: Deleting consensus metadata
I20260806 01:07:01.777467 25352 catalog_manager.cc:4841] TS aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069): tablet 444d5a48a47d4ea4956c231a94724917 (table test-table [id=1908e1425785415291385920595147ed]) successfully deleted
I20260806 01:07:01.779589 26216 ts_tablet_manager.cc:1918] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:01.779917 26216 log.cc:1196] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/444d5a48a47d4ea4956c231a94724917
I20260806 01:07:01.780727 26216 ts_tablet_manager.cc:1939] T 444d5a48a47d4ea4956c231a94724917 P d55c9fca009740e998167d129795be0c: Deleting consensus metadata
I20260806 01:07:01.782763 25351 catalog_manager.cc:4841] TS d55c9fca009740e998167d129795be0c (127.22.63.66:36249): tablet 444d5a48a47d4ea4956c231a94724917 (table test-table [id=1908e1425785415291385920595147ed]) successfully deleted
I20260806 01:07:01.785055 26217 ts_tablet_manager.cc:1918] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:01.785290 26217 log.cc:1196] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/444d5a48a47d4ea4956c231a94724917
I20260806 01:07:01.785781 26217 ts_tablet_manager.cc:1939] T 444d5a48a47d4ea4956c231a94724917 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting consensus metadata
I20260806 01:07:01.787480 25353 catalog_manager.cc:4841] TS 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501): tablet 444d5a48a47d4ea4956c231a94724917 (table test-table [id=1908e1425785415291385920595147ed]) successfully deleted
I20260806 01:07:01.887956 26173 raft_consensus.cc:482] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:01.888274 26173 raft_consensus.cc:504] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.889127 26173 leader_election.cc:288] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:01.889842 25607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92c726fb1814ad88d2b18ce5a077dfa" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" is_pre_election: true
I20260806 01:07:01.889961 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92c726fb1814ad88d2b18ce5a077dfa" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c" is_pre_election: true
I20260806 01:07:01.890156 25607 raft_consensus.cc:2454] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:01.890192 26035 raft_consensus.cc:2454] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:01.890735 25710 leader_election.cc:302] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed9a11b02b64a128dab88810f826f7c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.891135 26173 raft_consensus.cc:2781] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:01.891256 26173 raft_consensus.cc:482] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:01.891377 26173 raft_consensus.cc:3037] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.893781 26173 raft_consensus.cc:504] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.894975 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92c726fb1814ad88d2b18ce5a077dfa" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c"
I20260806 01:07:01.894968 25607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a92c726fb1814ad88d2b18ce5a077dfa" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
I20260806 01:07:01.895321 25607 raft_consensus.cc:3037] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.895309 26035 raft_consensus.cc:3037] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:01.898502 26035 raft_consensus.cc:2454] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:01.898604 25607 raft_consensus.cc:2454] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:01.899499 25711 leader_election.cc:302] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:01.900024 26148 raft_consensus.cc:2781] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:01.900104 26173 leader_election.cc:288] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Requested vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:01.900223 26148 raft_consensus.cc:686] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 1 LEADER]: Becoming Leader. State: Replica: d55c9fca009740e998167d129795be0c, State: Running, Role: LEADER
I20260806 01:07:01.900508 26148 consensus_queue.cc:237] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:01.904728 25366 catalog_manager.cc:5495] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c reported cstate change: term changed from 0 to 1, leader changed from <none> to d55c9fca009740e998167d129795be0c (127.22.63.66). New cstate: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:01.965442 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.958635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46580 (local address 127.22.63.66:36249)
0806 01:07:01.959115 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.959121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.959139 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:07:01.959210 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.959215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.959318 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.959417 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.959954 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.961099 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.963097 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.963868 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.964026 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.964556 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.964714 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.964817 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.965008 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.965098 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360}
I20260806 01:07:01.985899 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.978477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36249 (local address 127.22.63.66:53167)
0806 01:07:01.978699 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:01.978728 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:01.978879 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:01.979279 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:01.979285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:01.979300 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:01.979591 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.979606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.981277 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.981283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.983028 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:01.983045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.984296 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.984304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:01.984531 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.984595 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.985186 (+ 591us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.985202 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 01:07:01.985297 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 01:07:01.985491 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 01:07:01.985951 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:01.978584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:53167 (local address 127.22.63.66:36249)
0806 01:07:01.978765 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:01.978773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:01.978796 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:07:01.979031 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:01.979036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:01.979126 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:01.979229 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:01.979796 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.981103 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.983225 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:01.984080 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:01.984268 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:01.984816 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:01.985036 (+ 220us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:01.985155 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:01.985357 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 01:07:01.985475 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":59}
I20260806 01:07:02.007618 26035 raft_consensus.cc:1261] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:02.007711 25607 raft_consensus.cc:1261] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:02.008657 26211 consensus_queue.cc:1035] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:02.009119 26173 consensus_queue.cc:1035] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:07:02.061200 26180 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:07:03.683096 25366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:03.683341 25366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:03.688175 25366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Sending DeleteTablet for 3 replicas of tablet a92c726fb1814ad88d2b18ce5a077dfa
I20260806 01:07:03.689244 25577 tablet_service.cc:1514] Processing DeleteTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:03 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40128
I20260806 01:07:03.689262 25776 tablet_service.cc:1514] Processing DeleteTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:03 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46560
I20260806 01:07:03.689625 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet a92c726fb1814ad88d2b18ce5a077dfa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:03 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:41174
I20260806 01:07:03.701776 26261 tablet_replica.cc:330] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: stopping tablet replica
I20260806 01:07:03.702281 26261 raft_consensus.cc:2229] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:03.703169 26261 raft_consensus.cc:2258] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:03.704281 26262 tablet_replica.cc:330] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: stopping tablet replica
I20260806 01:07:03.704507 26259 tablet_replica.cc:330] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: stopping tablet replica
I20260806 01:07:03.704535 26261 ts_tablet_manager.cc:1905] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:03.704521 26262 raft_consensus.cc:2229] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:03.704727 26259 raft_consensus.cc:2229] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:03.704784 26262 raft_consensus.cc:2258] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:03.704983 26259 raft_consensus.cc:2258] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:03.706142 26259 ts_tablet_manager.cc:1905] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:03.707863 26262 ts_tablet_manager.cc:1905] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:03.711575 26261 ts_tablet_manager.cc:1918] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:03.711720 26261 log.cc:1196] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/a92c726fb1814ad88d2b18ce5a077dfa
I20260806 01:07:03.712250 26261 ts_tablet_manager.cc:1939] T a92c726fb1814ad88d2b18ce5a077dfa P d55c9fca009740e998167d129795be0c: Deleting consensus metadata
I20260806 01:07:03.712834 26262 ts_tablet_manager.cc:1918] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:03.712986 26262 log.cc:1196] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/a92c726fb1814ad88d2b18ce5a077dfa
I20260806 01:07:03.713150 26259 ts_tablet_manager.cc:1918] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:03.713302 26259 log.cc:1196] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/a92c726fb1814ad88d2b18ce5a077dfa
I20260806 01:07:03.713526 26262 ts_tablet_manager.cc:1939] T a92c726fb1814ad88d2b18ce5a077dfa P aed9a11b02b64a128dab88810f826f7c: Deleting consensus metadata
I20260806 01:07:03.713573 25351 catalog_manager.cc:4841] TS d55c9fca009740e998167d129795be0c (127.22.63.66:36249): tablet a92c726fb1814ad88d2b18ce5a077dfa (table test-table [id=dd9802ccaf424e218ec247678408d52a]) successfully deleted
I20260806 01:07:03.713794 26259 ts_tablet_manager.cc:1939] T a92c726fb1814ad88d2b18ce5a077dfa P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting consensus metadata
I20260806 01:07:03.714768 25352 catalog_manager.cc:4841] TS aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069): tablet a92c726fb1814ad88d2b18ce5a077dfa (table test-table [id=dd9802ccaf424e218ec247678408d52a]) successfully deleted
I20260806 01:07:03.714799 25353 catalog_manager.cc:4841] TS 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501): tablet a92c726fb1814ad88d2b18ce5a077dfa (table test-table [id=dd9802ccaf424e218ec247678408d52a]) successfully deleted
Aug 06 01:07:03 dist-test-slave-7f01 krb5kdc[25302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978423, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:07:03 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978423, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:03.762240 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:03.747377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.22.63.126:40009)
0806 01:07:03.747978 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:03.747982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:03.748030 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:07:03.748087 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:03.748091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:03.748155 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:03.748243 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:03.749794 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:03.750614 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:03.752094 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:03.752558 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:03.752665 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:03.756880 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:03.756886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:03.756889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:03.756924 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:03.759727 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:03.760456 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:03.760460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:03.760462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:03.760591 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:03.760973 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:03.760976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:03.760978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:03.761248 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:03.761436 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:03.761781 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 01:07:03.761962 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":306,"threads_started":1}
I20260806 01:07:03.766186 25366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:03.766809 25366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:03.774243 25577 tablet_service.cc:1467] Processing CreateTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 (DEFAULT_TABLE table=test-table [id=19b91282258f48039f3143d124e21357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:03.774581 25999 tablet_service.cc:1467] Processing CreateTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 (DEFAULT_TABLE table=test-table [id=19b91282258f48039f3143d124e21357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:03.774770 25577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af9443dd8be41e8b64a1c7bad0f0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:03.775024 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af9443dd8be41e8b64a1c7bad0f0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:03.780246 25776 tablet_service.cc:1467] Processing CreateTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 (DEFAULT_TABLE table=test-table [id=19b91282258f48039f3143d124e21357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:03.780709 25776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af9443dd8be41e8b64a1c7bad0f0302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:03.781896 26273 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Bootstrap starting.
I20260806 01:07:03.782820 26274 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Bootstrap starting.
I20260806 01:07:03.784392 26273 tablet_bootstrap.cc:654] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:03.784706 26274 tablet_bootstrap.cc:654] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:03.787035 26273 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: No bootstrap required, opened a new log
I20260806 01:07:03.787276 26273 ts_tablet_manager.cc:1397] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:03.788591 26273 raft_consensus.cc:348] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.788982 26273 raft_consensus.cc:374] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:03.789105 26273 raft_consensus.cc:729] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9a11b02b64a128dab88810f826f7c, State: Initialized, Role: FOLLOWER
I20260806 01:07:03.789427 26273 consensus_queue.cc:260] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.791322 26273 ts_tablet_manager.cc:1428] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 01:07:03.792049 26279 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Bootstrap starting.
I20260806 01:07:03.794025 26279 tablet_bootstrap.cc:654] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:03.794346 26274 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: No bootstrap required, opened a new log
I20260806 01:07:03.794546 26274 ts_tablet_manager.cc:1397] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260806 01:07:03.795487 26274 raft_consensus.cc:348] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.795790 26274 raft_consensus.cc:374] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:03.795931 26274 raft_consensus.cc:729] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Initialized, Role: FOLLOWER
I20260806 01:07:03.796195 26274 consensus_queue.cc:260] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.796423 26279 tablet_bootstrap.cc:492] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: No bootstrap required, opened a new log
I20260806 01:07:03.796587 26279 ts_tablet_manager.cc:1397] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 01:07:03.797231 26279 raft_consensus.cc:348] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.797451 26279 raft_consensus.cc:374] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:03.797551 26279 raft_consensus.cc:729] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55c9fca009740e998167d129795be0c, State: Initialized, Role: FOLLOWER
I20260806 01:07:03.797796 26279 consensus_queue.cc:260] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.798405 26279 ts_tablet_manager.cc:1428] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 01:07:03.799113 26274 ts_tablet_manager.cc:1428] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260806 01:07:03.899549 26173 raft_consensus.cc:482] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:03.899879 26173 raft_consensus.cc:504] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.900635 26173 leader_election.cc:288] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:03.901288 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af9443dd8be41e8b64a1c7bad0f0302" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c" is_pre_election: true
I20260806 01:07:03.901291 25605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af9443dd8be41e8b64a1c7bad0f0302" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" is_pre_election: true
I20260806 01:07:03.901561 26035 raft_consensus.cc:2454] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:03.901618 25605 raft_consensus.cc:2454] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 0.
I20260806 01:07:03.902108 25710 leader_election.cc:302] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed9a11b02b64a128dab88810f826f7c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:03.902415 26173 raft_consensus.cc:2781] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:03.902547 26173 raft_consensus.cc:482] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:03.902675 26173 raft_consensus.cc:3037] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:03.904843 26173 raft_consensus.cc:504] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.905375 26173 leader_election.cc:288] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Requested vote from peers 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:03.905951 26035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af9443dd8be41e8b64a1c7bad0f0302" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c"
I20260806 01:07:03.905987 25605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af9443dd8be41e8b64a1c7bad0f0302" candidate_uuid: "d55c9fca009740e998167d129795be0c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c"
I20260806 01:07:03.906213 25605 raft_consensus.cc:3037] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:03.906209 26035 raft_consensus.cc:3037] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:03.909201 25605 raft_consensus.cc:2454] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:03.909300 26035 raft_consensus.cc:2454] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55c9fca009740e998167d129795be0c in term 1.
I20260806 01:07:03.909641 25711 leader_election.cc:302] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:03.909960 26173 raft_consensus.cc:2781] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:03.910117 26173 raft_consensus.cc:686] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 1 LEADER]: Becoming Leader. State: Replica: d55c9fca009740e998167d129795be0c, State: Running, Role: LEADER
I20260806 01:07:03.910429 26173 consensus_queue.cc:237] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:03.913064 25365 catalog_manager.cc:5495] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c reported cstate change: term changed from 0 to 1, leader changed from <none> to d55c9fca009740e998167d129795be0c (127.22.63.66). New cstate: current_term: 1 leader_uuid: "d55c9fca009740e998167d129795be0c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:03.958014 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:03.952180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.22.63.66:36249)
0806 01:07:03.952764 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:03.952768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:03.952789 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:07:03.952841 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:03.952844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:03.952905 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:03.953005 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:03.953474 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:03.954273 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:03.955684 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:03.956291 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:03.956399 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:03.957061 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:03.957259 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:03.957387 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:03.957616 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 01:07:03.957691 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":344,"threads_started":1}
I20260806 01:07:03.968590 26035 raft_consensus.cc:1261] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:03.968741 25605 raft_consensus.cc:1261] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Refusing update from remote peer d55c9fca009740e998167d129795be0c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:03.969164 26173 consensus_queue.cc:1035] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:03.969656 26240 consensus_queue.cc:1035] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:04.014295 25366 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:04.014588 25366 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:04.017778 25366 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Sending DeleteTablet for 3 replicas of tablet 8af9443dd8be41e8b64a1c7bad0f0302
I20260806 01:07:04.018622 25577 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:04 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40128
I20260806 01:07:04.018901 26259 tablet_replica.cc:330] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: stopping tablet replica
I20260806 01:07:04.018831 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:04 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:41174
I20260806 01:07:04.019102 26259 raft_consensus.cc:2229] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:04.019311 26259 raft_consensus.cc:2258] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:04.019421 25777 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af9443dd8be41e8b64a1c7bad0f0302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:04 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46560
I20260806 01:07:04.019603 26262 tablet_replica.cc:330] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: stopping tablet replica
I20260806 01:07:04.019732 26261 tablet_replica.cc:330] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: stopping tablet replica
I20260806 01:07:04.019822 26262 raft_consensus.cc:2229] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:04.020013 26262 raft_consensus.cc:2258] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:04.020007 26261 raft_consensus.cc:2229] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:04.020323 26261 raft_consensus.cc:2258] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:04.021193 26262 ts_tablet_manager.cc:1905] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:04.021446 26261 ts_tablet_manager.cc:1905] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:04.025913 26261 ts_tablet_manager.cc:1918] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:04.025928 26262 ts_tablet_manager.cc:1918] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:04.026077 26261 log.cc:1196] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/8af9443dd8be41e8b64a1c7bad0f0302
I20260806 01:07:04.026103 26262 log.cc:1196] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/8af9443dd8be41e8b64a1c7bad0f0302
I20260806 01:07:04.026201 26259 ts_tablet_manager.cc:1905] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:04.026513 26262 ts_tablet_manager.cc:1939] T 8af9443dd8be41e8b64a1c7bad0f0302 P aed9a11b02b64a128dab88810f826f7c: Deleting consensus metadata
I20260806 01:07:04.026613 26261 ts_tablet_manager.cc:1939] T 8af9443dd8be41e8b64a1c7bad0f0302 P d55c9fca009740e998167d129795be0c: Deleting consensus metadata
I20260806 01:07:04.027769 25352 catalog_manager.cc:4841] TS aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069): tablet 8af9443dd8be41e8b64a1c7bad0f0302 (table test-table [id=19b91282258f48039f3143d124e21357]) successfully deleted
I20260806 01:07:04.027709 25366 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:04.028672 25366 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:04.029901 25351 catalog_manager.cc:4841] TS d55c9fca009740e998167d129795be0c (127.22.63.66:36249): tablet 8af9443dd8be41e8b64a1c7bad0f0302 (table test-table [id=19b91282258f48039f3143d124e21357]) successfully deleted
I20260806 01:07:04.032366 26259 ts_tablet_manager.cc:1918] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:04.032562 26259 log.cc:1196] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/8af9443dd8be41e8b64a1c7bad0f0302
I20260806 01:07:04.033111 26259 ts_tablet_manager.cc:1939] T 8af9443dd8be41e8b64a1c7bad0f0302 P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting consensus metadata
I20260806 01:07:04.034241 25353 catalog_manager.cc:4841] TS 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501): tablet 8af9443dd8be41e8b64a1c7bad0f0302 (table test-table [id=19b91282258f48039f3143d124e21357]) successfully deleted
I20260806 01:07:04.035677 25577 tablet_service.cc:1467] Processing CreateTablet for tablet 5489da173d3c48c4a42d57bb603d62cc (DEFAULT_TABLE table=test-table [id=18628e4297c041e9aac49ad77c4574de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:04.036158 25577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5489da173d3c48c4a42d57bb603d62cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:04.036201 25999 tablet_service.cc:1467] Processing CreateTablet for tablet 5489da173d3c48c4a42d57bb603d62cc (DEFAULT_TABLE table=test-table [id=18628e4297c041e9aac49ad77c4574de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:04.036660 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5489da173d3c48c4a42d57bb603d62cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:04.036957 25777 tablet_service.cc:1467] Processing CreateTablet for tablet 5489da173d3c48c4a42d57bb603d62cc (DEFAULT_TABLE table=test-table [id=18628e4297c041e9aac49ad77c4574de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:04.037392 25777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5489da173d3c48c4a42d57bb603d62cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:04.044381 26279 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Bootstrap starting.
I20260806 01:07:04.044703 26273 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Bootstrap starting.
I20260806 01:07:04.045104 26274 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Bootstrap starting.
I20260806 01:07:04.046193 26279 tablet_bootstrap.cc:654] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:04.046362 26273 tablet_bootstrap.cc:654] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:04.046483 26274 tablet_bootstrap.cc:654] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:04.049154 26279 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: No bootstrap required, opened a new log
I20260806 01:07:04.049403 26279 ts_tablet_manager.cc:1397] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260806 01:07:04.050433 26279 raft_consensus.cc:348] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.050766 26279 raft_consensus.cc:374] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:04.050907 26279 raft_consensus.cc:729] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55c9fca009740e998167d129795be0c, State: Initialized, Role: FOLLOWER
I20260806 01:07:04.051124 26273 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: No bootstrap required, opened a new log
I20260806 01:07:04.051165 26279 consensus_queue.cc:260] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.051308 26273 ts_tablet_manager.cc:1397] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 01:07:04.052639 26274 tablet_bootstrap.cc:492] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: No bootstrap required, opened a new log
I20260806 01:07:04.052662 26279 ts_tablet_manager.cc:1428] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260806 01:07:04.052901 26274 ts_tablet_manager.cc:1397] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260806 01:07:04.052207 26273 raft_consensus.cc:348] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.053082 26273 raft_consensus.cc:374] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:04.053191 26273 raft_consensus.cc:729] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed9a11b02b64a128dab88810f826f7c, State: Initialized, Role: FOLLOWER
I20260806 01:07:04.053442 26273 consensus_queue.cc:260] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.053937 26273 ts_tablet_manager.cc:1428] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260806 01:07:04.053825 26274 raft_consensus.cc:348] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.054076 26274 raft_consensus.cc:374] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:04.054157 26274 raft_consensus.cc:729] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Initialized, Role: FOLLOWER
I20260806 01:07:04.054399 26274 consensus_queue.cc:260] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.055004 26274 ts_tablet_manager.cc:1428] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 01:07:04.106894 26284 raft_consensus.cc:482] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:04.107215 26284 raft_consensus.cc:504] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.108060 26284 leader_election.cc:288] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55c9fca009740e998167d129795be0c (127.22.63.66:36249), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:04.108748 26033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5489da173d3c48c4a42d57bb603d62cc" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c" is_pre_election: true
I20260806 01:07:04.108773 25822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5489da173d3c48c4a42d57bb603d62cc" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55c9fca009740e998167d129795be0c" is_pre_election: true
I20260806 01:07:04.109021 26033 raft_consensus.cc:2454] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 0.
I20260806 01:07:04.109058 25822 raft_consensus.cc:2454] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 0.
I20260806 01:07:04.109561 25515 leader_election.cc:302] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, aed9a11b02b64a128dab88810f826f7c; no voters:
I20260806 01:07:04.109897 26284 raft_consensus.cc:2781] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:04.110006 26284 raft_consensus.cc:482] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:04.110074 26284 raft_consensus.cc:3037] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:04.112350 26284 raft_consensus.cc:504] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.113073 26284 leader_election.cc:288] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 election: Requested vote from peers d55c9fca009740e998167d129795be0c (127.22.63.66:36249), aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069)
I20260806 01:07:04.113536 25822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5489da173d3c48c4a42d57bb603d62cc" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55c9fca009740e998167d129795be0c"
I20260806 01:07:04.113636 26033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5489da173d3c48c4a42d57bb603d62cc" candidate_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed9a11b02b64a128dab88810f826f7c"
I20260806 01:07:04.113781 25822 raft_consensus.cc:3037] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:04.113883 26033 raft_consensus.cc:3037] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:04.116974 25822 raft_consensus.cc:2454] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 1.
I20260806 01:07:04.116974 26033 raft_consensus.cc:2454] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f98199ffebb4f9a9ef39f44f1f5984c in term 1.
I20260806 01:07:04.117666 25515 leader_election.cc:302] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f98199ffebb4f9a9ef39f44f1f5984c, aed9a11b02b64a128dab88810f826f7c, d55c9fca009740e998167d129795be0c; no voters:
I20260806 01:07:04.118173 26284 raft_consensus.cc:2781] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:04.118577 26284 raft_consensus.cc:686] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 LEADER]: Becoming Leader. State: Replica: 4f98199ffebb4f9a9ef39f44f1f5984c, State: Running, Role: LEADER
I20260806 01:07:04.118933 26284 consensus_queue.cc:237] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } }
I20260806 01:07:04.121788 25365 catalog_manager.cc:5495] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65). New cstate: current_term: 1 leader_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f98199ffebb4f9a9ef39f44f1f5984c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:04.150286 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:04.144130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.22.63.65:46501)
0806 01:07:04.144826 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:04.144832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:04.144857 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:07:04.144929 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:04.144936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:04.145052 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:04.145151 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:04.145586 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:04.146784 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:04.148142 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:04.148813 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:04.148949 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:04.149396 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:04.149530 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:04.149616 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:04.149791 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 01:07:04.149875 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":404,"threads_started":1}
I20260806 01:07:04.160851 26033 raft_consensus.cc:1261] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Refusing update from remote peer 4f98199ffebb4f9a9ef39f44f1f5984c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:04.161000 25822 raft_consensus.cc:1261] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Refusing update from remote peer 4f98199ffebb4f9a9ef39f44f1f5984c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:04.161512 26295 consensus_queue.cc:1035] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed9a11b02b64a128dab88810f826f7c" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:04.161936 26284 consensus_queue.cc:1035] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55c9fca009740e998167d129795be0c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:07:04.184238 26237 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 01:07:05.776237 25368 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:05.776465 25368 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:05.780614 25368 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e7df64e2459b42398077e22362efce35: Sending DeleteTablet for 3 replicas of tablet 5489da173d3c48c4a42d57bb603d62cc
I20260806 01:07:05.781498 25577 tablet_service.cc:1514] Processing DeleteTablet for tablet 5489da173d3c48c4a42d57bb603d62cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:05 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40128
I20260806 01:07:05.781637 25777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5489da173d3c48c4a42d57bb603d62cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:05 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46560
I20260806 01:07:05.781697 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet 5489da173d3c48c4a42d57bb603d62cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:05 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:41174
I20260806 01:07:05.782598 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25488
I20260806 01:07:05.792227 26337 tablet_replica.cc:330] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: stopping tablet replica
I20260806 01:07:05.792476 26340 tablet_replica.cc:330] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: stopping tablet replica
I20260806 01:07:05.792631 26337 raft_consensus.cc:2229] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:05.792788 26340 raft_consensus.cc:2229] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:05.793105 26340 raft_consensus.cc:2258] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:05.793313 26337 raft_consensus.cc:2258] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:05.793917 26338 tablet_replica.cc:330] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: stopping tablet replica
I20260806 01:07:05.794137 26338 raft_consensus.cc:2229] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:05.794221 26340 ts_tablet_manager.cc:1905] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:05.794430 26338 raft_consensus.cc:2258] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:05.795071 26337 ts_tablet_manager.cc:1905] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:05.795553 26338 ts_tablet_manager.cc:1905] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:07:05 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978423, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:05.802160 26338 ts_tablet_manager.cc:1918] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:05.802232 26340 ts_tablet_manager.cc:1918] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:05.802381 26340 log.cc:1196] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/5489da173d3c48c4a42d57bb603d62cc
I20260806 01:07:05.802440 26338 log.cc:1196] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/5489da173d3c48c4a42d57bb603d62cc
I20260806 01:07:05.802644 26337 ts_tablet_manager.cc:1918] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 01:07:05.802871 26337 log.cc:1196] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/5489da173d3c48c4a42d57bb603d62cc
I20260806 01:07:05.802989 26340 ts_tablet_manager.cc:1939] T 5489da173d3c48c4a42d57bb603d62cc P aed9a11b02b64a128dab88810f826f7c: Deleting consensus metadata
I20260806 01:07:05.802986 26338 ts_tablet_manager.cc:1939] T 5489da173d3c48c4a42d57bb603d62cc P d55c9fca009740e998167d129795be0c: Deleting consensus metadata
I20260806 01:07:05.803571 26337 ts_tablet_manager.cc:1939] T 5489da173d3c48c4a42d57bb603d62cc P 4f98199ffebb4f9a9ef39f44f1f5984c: Deleting consensus metadata
I20260806 01:07:05.804162 25352 catalog_manager.cc:4841] TS aed9a11b02b64a128dab88810f826f7c (127.22.63.67:41069): tablet 5489da173d3c48c4a42d57bb603d62cc (table test-table [id=18628e4297c041e9aac49ad77c4574de]) successfully deleted
I20260806 01:07:05.804229 25351 catalog_manager.cc:4841] TS d55c9fca009740e998167d129795be0c (127.22.63.66:36249): tablet 5489da173d3c48c4a42d57bb603d62cc (table test-table [id=18628e4297c041e9aac49ad77c4574de]) successfully deleted
I20260806 01:07:05.804781 25353 catalog_manager.cc:4841] TS 4f98199ffebb4f9a9ef39f44f1f5984c (127.22.63.65:46501): tablet 5489da173d3c48c4a42d57bb603d62cc (table test-table [id=18628e4297c041e9aac49ad77c4574de]) successfully deleted
I20260806 01:07:05.808622 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:05.784004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.22.63.65:46501)
0806 01:07:05.785044 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:05.785053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:05.785820 (+ 767us) server_negotiation.cc:407] Connection header received
0806 01:07:05.785904 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:05.785912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:05.786026 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:05.786149 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:05.789004 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.790783 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.793553 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.794123 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.794264 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:05.800211 (+ 5947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:05.800235 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:05.800262 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:05.800311 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:05.805335 (+ 5024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.806338 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.806346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.806348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.806505 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.807024 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.807030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.807036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.807466 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:05.807715 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:05.808151 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 01:07:05.808338 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":633,"threads_started":1}
W20260806 01:07:05.809386 25677 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40154
W20260806 01:07:05.810036 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25488: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:05.810249 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25488
I20260806 01:07:05.846834 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25687
Aug 06 01:07:05 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978423, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:07:05.863373 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:05.847696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.22.63.66:36249)
0806 01:07:05.848297 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:05.848301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:05.848318 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:07:05.848383 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:05.848387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:05.848447 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:05.848541 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:05.849838 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.850605 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.852005 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.852521 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.852622 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:05.857235 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:05.857250 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:05.857277 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:05.857320 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:05.860449 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.861196 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.861205 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.861207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.861339 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.861750 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.861754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.861756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.862128 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:05.862426 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:05.862754 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 01:07:05.863002 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
W20260806 01:07:05.864149 25882 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46594
W20260806 01:07:05.864833 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:05.865010 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25687
I20260806 01:07:05.906883 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25897
Aug 06 01:07:05 dist-test-slave-7f01 krb5kdc[25302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978423, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:07:05.922448 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:05.907655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.22.63.67:41069)
0806 01:07:05.908250 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:05.908254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:05.908271 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:07:05.908326 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:05.908330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:05.908389 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:05.908484 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:05.909827 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.910593 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.911902 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:05.912280 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:05.912381 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:05.916698 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:05.916713 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:05.916731 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:05.916767 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:05.919595 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.920320 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.920328 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.920330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.920480 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:05.920879 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:05.920883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:05.920885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:05.921249 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:05.921541 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:05.921871 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 01:07:05.922108 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":376,"threads_started":1}
W20260806 01:07:05.923121 26100 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41176
W20260806 01:07:05.923810 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25897: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:05.924014 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25897
I20260806 01:07:05.947947 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25320
I20260806 01:07:05.949199 25437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:06.068017 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25320
2026-08-06T01:07:06Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8720 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:06 dist-test-slave-7f01 krb5kdc[26349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:06 dist-test-slave-7f01 krb5kdc[26349](info): set up 2 sockets
Aug 06 01:07:06 dist-test-slave-7f01 krb5kdc[26349](info): commencing operation
krb5kdc: starting...
W20260806 01:07:07.373827 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:07 dist-test-slave-7f01 krb5kdc[26349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:07.607344 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37745
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:07.945933 26372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:07.946244 26372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:07.946408 26372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:07.953639 26372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:07.953747 26372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:07.953848 26372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:07.953958 26372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:07.954041 26372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:07.963971 26372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:07.965380 26372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:07.966744 26372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:07.979312 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:07.979476 26372 server_base.cc:1034] running on GCE node
W20260806 01:07:07.979342 26382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:07.979390 26379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:07.980734 26372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:07.982225 26372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:07.983474 26372 hybrid_clock.cc:648] HybridClock initialized: now 1785978427983434 us; error 53 us; skew 500 ppm
I20260806 01:07:07.983894 26372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:07 dist-test-slave-7f01 krb5kdc[26349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978427, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:07.987210 26372 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:07.988718 26372 webserver.cc:460] Webserver started at http://127.22.63.126:40375/ using document root <none> and password file <none>
I20260806 01:07:07.989403 26372 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:07.989502 26372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:07.989768 26372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:07.993194 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "64b3c2527fab4cc48f8e7f47569befea"
format_stamp: "Formatted at 2026-08-06 01:07:07 on dist-test-slave-7f01"
server_key: "bbe772a7df446ea315a7bd069d006aab"
server_key_iv: "c7a7a93a438a282e2d9f97ecb9fe9d00"
server_key_version: "encryptionkey@0"
I20260806 01:07:07.993891 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "64b3c2527fab4cc48f8e7f47569befea"
format_stamp: "Formatted at 2026-08-06 01:07:07 on dist-test-slave-7f01"
server_key: "bbe772a7df446ea315a7bd069d006aab"
server_key_iv: "c7a7a93a438a282e2d9f97ecb9fe9d00"
server_key_version: "encryptionkey@0"
I20260806 01:07:07.999019 26372 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:08.002430 26394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:08.003710 26372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:08.003968 26372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "64b3c2527fab4cc48f8e7f47569befea"
format_stamp: "Formatted at 2026-08-06 01:07:07 on dist-test-slave-7f01"
server_key: "bbe772a7df446ea315a7bd069d006aab"
server_key_iv: "c7a7a93a438a282e2d9f97ecb9fe9d00"
server_key_version: "encryptionkey@0"
I20260806 01:07:08.004177 26372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:08.023056 26372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:08.027208 26372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:08.027525 26372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:08.046473 26372 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37745
I20260806 01:07:08.046501 26471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37745 every 8 connection(s)
I20260806 01:07:08.047937 26372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:08.050722 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 26372
I20260806 01:07:08.050982 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:08.051832 22781 external_mini_cluster.cc:1428] Setting key 91cd588df56e44893f8d972cb72a4081
I20260806 01:07:08.052404 26472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:08.068163 26472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea: Bootstrap starting.
Aug 06 01:07:08 dist-test-slave-7f01 krb5kdc[26349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:08.071676 26472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:08.072837 26472 log.cc:824] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:08.075822 26472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea: No bootstrap required, opened a new log
I20260806 01:07:08.076949 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:08.053967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.22.63.126:37745)
0806 01:07:08.055136 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:08.055151 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:08.055208 (+ 57us) server_negotiation.cc:407] Connection header received
0806 01:07:08.057850 (+ 2642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:08.057874 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:08.058278 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:08.058626 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:08.060505 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.061777 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.063638 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.064448 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.064610 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:08.069453 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:08.069464 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:08.069481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:08.069528 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:08.072799 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:08.073564 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:08.073571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:08.073577 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:08.073756 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:08.074151 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:08.074157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:08.074160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:08.074715 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:08.075023 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:08.075473 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 01:07:08.075969 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":675,"threads_started":1}
I20260806 01:07:08.083447 26472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } }
I20260806 01:07:08.083873 26472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:08.084005 26472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b3c2527fab4cc48f8e7f47569befea, State: Initialized, Role: FOLLOWER
I20260806 01:07:08.084583 26472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } }
I20260806 01:07:08.084808 26472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:08.084951 26472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:08.085125 26472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:08.087500 26472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } }
I20260806 01:07:08.088057 26472 leader_election.cc:302] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64b3c2527fab4cc48f8e7f47569befea; no voters:
I20260806 01:07:08.088661 26472 leader_election.cc:288] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:08.088814 26480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:08.089540 26480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [term 1 LEADER]: Becoming Leader. State: Replica: 64b3c2527fab4cc48f8e7f47569befea, State: Running, Role: LEADER
I20260806 01:07:08.090076 26480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } }
I20260806 01:07:08.090772 26472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:08.092985 26481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64b3c2527fab4cc48f8e7f47569befea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } } }
I20260806 01:07:08.093341 26481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:08.094803 26482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64b3c2527fab4cc48f8e7f47569befea. Latest consensus state: current_term: 1 leader_uuid: "64b3c2527fab4cc48f8e7f47569befea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64b3c2527fab4cc48f8e7f47569befea" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37745 } } }
I20260806 01:07:08.095070 26482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:08.095139 26486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:08.103230 26486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:08.113076 26486 catalog_manager.cc:1347] Generated new cluster ID: a06f2b3314844e52b7315e502092ceff
I20260806 01:07:08.113215 26486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:08.123669 26486 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:08.124967 26486 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:08.142511 26486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64b3c2527fab4cc48f8e7f47569befea: Generated new TSK 0
I20260806 01:07:08.143363 26486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:07:08.205302 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:08.573851 26510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:08.574359 26510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:08.574638 26510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:08.584153 26510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:08.584275 26510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:08.584391 26510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:08.594910 26510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:08.596439 26510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:08.597990 26510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:08.613561 26516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:08.613538 26519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:08.613492 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:08.614535 26510 server_base.cc:1034] running on GCE node
I20260806 01:07:08.615315 26510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:08.616443 26510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:08.617674 26510 hybrid_clock.cc:648] HybridClock initialized: now 1785978428617617 us; error 46 us; skew 500 ppm
I20260806 01:07:08.618027 26510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:08 dist-test-slave-7f01 krb5kdc[26349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:08.621608 26510 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:07:08.623258 26510 webserver.cc:460] Webserver started at http://127.22.63.65:44885/ using document root <none> and password file <none>
I20260806 01:07:08.624099 26510 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:08.624254 26510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:08.624585 26510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:08.627696 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "a5205a174fc044459c0966e095f1885a"
format_stamp: "Formatted at 2026-08-06 01:07:08 on dist-test-slave-7f01"
server_key: "55b2148d2319c3db9a6f8a281db28b98"
server_key_iv: "82e55b014f40e7573dbe0b2995d53a1d"
server_key_version: "encryptionkey@0"
I20260806 01:07:08.628641 26510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "a5205a174fc044459c0966e095f1885a"
format_stamp: "Formatted at 2026-08-06 01:07:08 on dist-test-slave-7f01"
server_key: "55b2148d2319c3db9a6f8a281db28b98"
server_key_iv: "82e55b014f40e7573dbe0b2995d53a1d"
server_key_version: "encryptionkey@0"
I20260806 01:07:08.634359 26510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:07:08.638088 26529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:08.639698 26510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:08.639920 26510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "a5205a174fc044459c0966e095f1885a"
format_stamp: "Formatted at 2026-08-06 01:07:08 on dist-test-slave-7f01"
server_key: "55b2148d2319c3db9a6f8a281db28b98"
server_key_iv: "82e55b014f40e7573dbe0b2995d53a1d"
server_key_version: "encryptionkey@0"
I20260806 01:07:08.640146 26510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:08.658946 26510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:08.663314 26510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:08.663650 26510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:08.664839 26510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:08.666347 26510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:08.666456 26510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:08.666615 26510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:08.666697 26510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:08.702455 26510 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:33881
I20260806 01:07:08.702507 26707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:33881 every 8 connection(s)
I20260806 01:07:08.703864 26510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:08.710211 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 26510
I20260806 01:07:08.710469 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:08.711303 22781 external_mini_cluster.cc:1428] Setting key 7f983ea70933e9f1b045a0023798a1b2
Aug 06 01:07:08 dist-test-slave-7f01 krb5kdc[26349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978428, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:08.726760 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:08.706552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:41847 (local address 127.22.63.126:37745)
0806 01:07:08.707077 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:08.707084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:08.708226 (+ 1142us) server_negotiation.cc:407] Connection header received
0806 01:07:08.710400 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:08.710411 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:08.710541 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:08.710662 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:08.712925 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.713904 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.715531 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.716006 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.716155 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:08.720250 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:08.720260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:08.720268 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:08.720316 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:08.723415 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:08.724264 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:08.724271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:08.724279 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:08.724393 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:08.724889 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:08.724894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:08.724898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:08.725172 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:08.725328 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:08.726111 (+ 783us) server_negotiation.cc:299] Negotiation successful
0806 01:07:08.726348 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260806 01:07:08.727926 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:08.707205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37745 (local address 127.22.63.65:41847)
0806 01:07:08.708068 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:08.708114 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:08.710003 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:08.710975 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:08.710989 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:08.711826 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:08.712643 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:08.712664 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.714088 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.714092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:08.715370 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:08.715382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:08.716283 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:08.716291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:08.716537 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:08.717268 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:08.717301 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:08.719987 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:08.723577 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:08.723586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:08.723602 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:08.724131 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:08.724561 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:08.724566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:08.724569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:08.724764 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:08.725307 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:08.725315 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:08.725724 (+ 409us) client_negotiation.cc:769] Sending connection context
0806 01:07:08.726024 (+ 300us) client_negotiation.cc:240] Negotiation successful
0806 01:07:08.726576 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":281,"threads_started":1}
I20260806 01:07:08.729710 26708 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37745
I20260806 01:07:08.730031 26708 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:08.731158 26708 heartbeater.cc:511] Master 127.22.63.126:37745 requested a full tablet report, sending...
I20260806 01:07:08.733904 26422 ts_manager.cc:194] Registered new tserver with Master: a5205a174fc044459c0966e095f1885a (127.22.63.65:33881)
I20260806 01:07:08.736208 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:41847
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
W20260806 01:07:09.583405 22781 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.63.66: real 0.869s user 0.001s sys 0.016s
I20260806 01:07:09.584307 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:07:09.739377 26708 heartbeater.cc:503] Master 127.22.63.126:37745 was elected leader, sending a full tablet report...
W20260806 01:07:09.935377 26717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:09.935688 26717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:09.935877 26717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:09.943235 26717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:09.943384 26717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:09.943589 26717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:09.953742 26717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:09.955183 26717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:09.956634 26717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:09.974654 26725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:09.974607 26731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:09.975210 26726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:09.975729 26717 server_base.cc:1034] running on GCE node
I20260806 01:07:09.976442 26717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:09.977412 26717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:09.978663 26717 hybrid_clock.cc:648] HybridClock initialized: now 1785978429978609 us; error 85 us; skew 500 ppm
I20260806 01:07:09.979012 26717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:09 dist-test-slave-7f01 krb5kdc[26349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978429, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:09.982358 26717 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:09.983948 26717 webserver.cc:460] Webserver started at http://127.22.63.66:44687/ using document root <none> and password file <none>
I20260806 01:07:09.984638 26717 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:09.984738 26717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:09.985001 26717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:09.988142 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "62775f38ab3f4d15a0712b08f53a98fb"
format_stamp: "Formatted at 2026-08-06 01:07:09 on dist-test-slave-7f01"
server_key: "db58a3736ae69366f11393701d40bf64"
server_key_iv: "702086714ed944e0e92fe6069de5a705"
server_key_version: "encryptionkey@0"
I20260806 01:07:09.988899 26717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "62775f38ab3f4d15a0712b08f53a98fb"
format_stamp: "Formatted at 2026-08-06 01:07:09 on dist-test-slave-7f01"
server_key: "db58a3736ae69366f11393701d40bf64"
server_key_iv: "702086714ed944e0e92fe6069de5a705"
server_key_version: "encryptionkey@0"
I20260806 01:07:09.994361 26717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:09.997862 26738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:09.999297 26717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:09.999449 26717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "62775f38ab3f4d15a0712b08f53a98fb"
format_stamp: "Formatted at 2026-08-06 01:07:09 on dist-test-slave-7f01"
server_key: "db58a3736ae69366f11393701d40bf64"
server_key_iv: "702086714ed944e0e92fe6069de5a705"
server_key_version: "encryptionkey@0"
I20260806 01:07:09.999624 26717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:10.023270 26717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:10.027874 26717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:10.028131 26717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:10.029093 26717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:10.030422 26717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:10.030506 26717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:10.030589 26717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:10.030634 26717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:10.073742 26717 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:40935
I20260806 01:07:10.073838 26893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:40935 every 8 connection(s)
I20260806 01:07:10.075088 26717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:10.080788 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 26717
I20260806 01:07:10.081001 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:10.081619 22781 external_mini_cluster.cc:1428] Setting key f172895940ccb94cdb39b95a376a954e
Aug 06 01:07:10 dist-test-slave-7f01 krb5kdc[26349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978429, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:10.097764 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:10.077784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:57107 (local address 127.22.63.126:37745)
0806 01:07:10.078247 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:10.078252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:10.079088 (+ 836us) server_negotiation.cc:407] Connection header received
0806 01:07:10.081175 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:10.081183 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:10.081278 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:10.081406 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:10.083113 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.084079 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.085742 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.086525 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.086675 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:10.090475 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:10.090484 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:10.090492 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:10.090543 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:10.093837 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:10.094908 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:10.094912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:10.094914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:10.095044 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:10.095586 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:10.095590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:10.095593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:10.096018 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:10.096160 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:10.097223 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0806 01:07:10.097412 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":245,"threads_started":1}
I20260806 01:07:10.099436 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:10.078195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37745 (local address 127.22.63.66:57107)
0806 01:07:10.078917 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:10.078956 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:10.080801 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:10.081605 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:10.081613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:10.082219 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:10.082925 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:10.082943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.084253 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.084257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:10.085580 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:10.085593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.086732 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.086737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:10.086948 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:10.087730 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:10.087806 (+ 76us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:10.090196 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:10.094020 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:10.094033 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:10.094057 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:10.094748 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:10.095161 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:10.095169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:10.095175 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:10.095437 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:10.096164 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:10.096175 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:10.096739 (+ 564us) client_negotiation.cc:769] Sending connection context
0806 01:07:10.097085 (+ 346us) client_negotiation.cc:240] Negotiation successful
0806 01:07:10.097757 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":276,"threads_started":1}
I20260806 01:07:10.101143 26894 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37745
I20260806 01:07:10.101480 26894 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:10.102250 26894 heartbeater.cc:511] Master 127.22.63.126:37745 requested a full tablet report, sending...
I20260806 01:07:10.104362 26422 ts_manager.cc:194] Registered new tserver with Master: 62775f38ab3f4d15a0712b08f53a98fb (127.22.63.66:40935)
I20260806 01:07:10.105358 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:57107
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
W20260806 01:07:10.381983 22781 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.63.67: real 0.297s user 0.000s sys 0.016s
I20260806 01:07:10.382807 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:10.730139 26903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:10.730445 26903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:10.730543 26903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:10.737900 26903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:10.738024 26903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:10.738140 26903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:10.748430 26903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37745
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:10.749755 26903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:10.751298 26903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:10.765254 26911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:10.765276 26912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:10.765218 26914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:10.766103 26903 server_base.cc:1034] running on GCE node
I20260806 01:07:10.766880 26903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:10.767892 26903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:10.769163 26903 hybrid_clock.cc:648] HybridClock initialized: now 1785978430769105 us; error 45 us; skew 500 ppm
I20260806 01:07:10.769500 26903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:10 dist-test-slave-7f01 krb5kdc[26349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978430, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:10.772800 26903 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:10.774369 26903 webserver.cc:460] Webserver started at http://127.22.63.67:40829/ using document root <none> and password file <none>
I20260806 01:07:10.775053 26903 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:10.775156 26903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:10.775429 26903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:10.778517 26903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "2c15d1ece9ac4fc987efe88aef2ae15f"
format_stamp: "Formatted at 2026-08-06 01:07:10 on dist-test-slave-7f01"
server_key: "948b5a1ce5e88f62b0031e0d5f353e9b"
server_key_iv: "04e3b28dde681b9c2056469bae0f1d3f"
server_key_version: "encryptionkey@0"
I20260806 01:07:10.779305 26903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "2c15d1ece9ac4fc987efe88aef2ae15f"
format_stamp: "Formatted at 2026-08-06 01:07:10 on dist-test-slave-7f01"
server_key: "948b5a1ce5e88f62b0031e0d5f353e9b"
server_key_iv: "04e3b28dde681b9c2056469bae0f1d3f"
server_key_version: "encryptionkey@0"
I20260806 01:07:10.784957 26903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:07:10.788879 26922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:10.790479 26903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:07:10.790633 26903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "2c15d1ece9ac4fc987efe88aef2ae15f"
format_stamp: "Formatted at 2026-08-06 01:07:10 on dist-test-slave-7f01"
server_key: "948b5a1ce5e88f62b0031e0d5f353e9b"
server_key_iv: "04e3b28dde681b9c2056469bae0f1d3f"
server_key_version: "encryptionkey@0"
I20260806 01:07:10.790882 26903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:10.803815 26903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:10.808593 26903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:10.808917 26903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:10.809930 26903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:10.811426 26903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:10.811538 26903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:10.811687 26903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:10.811781 26903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:10.847630 26903 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:39707
I20260806 01:07:10.847694 27089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:39707 every 8 connection(s)
I20260806 01:07:10.849119 26903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:10.857952 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 26903
I20260806 01:07:10.858196 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:10.858654 22781 external_mini_cluster.cc:1428] Setting key bea17036cfc2a5489a293427751f14b1
Aug 06 01:07:10 dist-test-slave-7f01 krb5kdc[26349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978430, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:10.871990 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:10.852113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:49983 (local address 127.22.63.126:37745)
0806 01:07:10.852680 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:10.852684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:10.853766 (+ 1082us) server_negotiation.cc:407] Connection header received
0806 01:07:10.855704 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:10.855709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:10.855856 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:10.856005 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:10.857707 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.858814 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.861156 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.861590 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.861761 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:10.865704 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:10.865710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:10.865714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:10.865751 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:10.868675 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:10.869507 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:10.869513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:10.869515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:10.869662 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:10.870185 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:10.870189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:10.870192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:10.870513 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:10.870648 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:10.871370 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 01:07:10.871563 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":294,"threads_started":1}
I20260806 01:07:10.873004 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:10.852820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37745 (local address 127.22.63.67:49983)
0806 01:07:10.853582 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:10.853623 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:10.855393 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:10.856256 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:10.856265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:10.856831 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:10.857449 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:10.857467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.859009 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.859014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:10.860990 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:10.861005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:10.861880 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:10.861890 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:10.862216 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:10.862969 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:10.862993 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:10.865445 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:10.868829 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:10.868838 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:10.868853 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:10.869325 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:10.869781 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:10.869785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:10.869798 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:10.870005 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:10.870635 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:10.870641 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:10.871000 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 01:07:10.871235 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 01:07:10.871681 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":295,"threads_started":1}
I20260806 01:07:10.874691 27091 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37745
I20260806 01:07:10.875234 27091 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:10.876150 27091 heartbeater.cc:511] Master 127.22.63.126:37745 requested a full tablet report, sending...
I20260806 01:07:10.877954 26422 ts_manager.cc:194] Registered new tserver with Master: 2c15d1ece9ac4fc987efe88aef2ae15f (127.22.63.67:39707)
I20260806 01:07:10.878903 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:49983
I20260806 01:07:10.888244 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:07:10.930151 27093 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:10.927268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.22.63.126:37745)
0806 01:07:10.927489 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:10.927494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:10.928055 (+ 561us) server_negotiation.cc:407] Connection header received
0806 01:07:10.928273 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:10.928284 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:10.928369 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:10.928491 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:10.929928 (+ 1437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60402: BlockingRecv error: recv got EOF from 127.0.0.1:60402 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:07:10.933997 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26510
W20260806 01:07:10.937204 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26510: Not authorized: Client connection negotiation failed: client connection to 127.22.63.65:33881: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:07:10.937505 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 26510
I20260806 01:07:10.963121 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26717
W20260806 01:07:10.966063 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26717: Not authorized: Client connection negotiation failed: client connection to 127.22.63.66:40935: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:07:10.966269 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 26717
I20260806 01:07:10.990505 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26903
W20260806 01:07:10.993938 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26903: Not authorized: Client connection negotiation failed: client connection to 127.22.63.67:39707: server requires authentication, but client does not have Kerberos credentials available
I20260806 01:07:10.994081 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 26903
I20260806 01:07:11.016753 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 26372
I20260806 01:07:11.018133 26466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:11.124835 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 26372
2026-08-06T01:07:11Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (5056 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:11 dist-test-slave-7f01 krb5kdc[27115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:11 dist-test-slave-7f01 krb5kdc[27115](info): set up 2 sockets
Aug 06 01:07:11 dist-test-slave-7f01 krb5kdc[27115](info): commencing operation
krb5kdc: starting...
W20260806 01:07:12.444118 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:12 dist-test-slave-7f01 krb5kdc[27115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:12.687336 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33571
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:35713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:13.027726 27133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:13.028061 27133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:13.028254 27133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:13.035552 27133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:13.035660 27133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:13.035751 27133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:13.035889 27133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:13.035971 27133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:13.045733 27133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:35713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:13.047093 27133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:13.048494 27133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:13.060214 27140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:13.060305 27143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:13.060261 27141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:13.061432 27133 server_base.cc:1034] running on GCE node
I20260806 01:07:13.062259 27133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:13.063706 27133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:13.064994 27133 hybrid_clock.cc:648] HybridClock initialized: now 1785978433064940 us; error 49 us; skew 500 ppm
I20260806 01:07:13.065372 27133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:13 dist-test-slave-7f01 krb5kdc[27115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978433, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:13.068604 27133 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:13.070183 27133 webserver.cc:460] Webserver started at http://127.22.63.126:43381/ using document root <none> and password file <none>
I20260806 01:07:13.070900 27133 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:13.071051 27133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:13.071367 27133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:13.075459 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "9aa058eb5bd64a4089c30bb5f61e64a8"
format_stamp: "Formatted at 2026-08-06 01:07:13 on dist-test-slave-7f01"
server_key: "62f4c84f4ee9963cb2b349f2910920a6"
server_key_iv: "3689e7e463e3c43821da2cd099eb83ba"
server_key_version: "encryptionkey@0"
I20260806 01:07:13.076359 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "9aa058eb5bd64a4089c30bb5f61e64a8"
format_stamp: "Formatted at 2026-08-06 01:07:13 on dist-test-slave-7f01"
server_key: "62f4c84f4ee9963cb2b349f2910920a6"
server_key_iv: "3689e7e463e3c43821da2cd099eb83ba"
server_key_version: "encryptionkey@0"
I20260806 01:07:13.081506 27133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:07:13.085003 27156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:13.086331 27133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:13.086516 27133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "9aa058eb5bd64a4089c30bb5f61e64a8"
format_stamp: "Formatted at 2026-08-06 01:07:13 on dist-test-slave-7f01"
server_key: "62f4c84f4ee9963cb2b349f2910920a6"
server_key_iv: "3689e7e463e3c43821da2cd099eb83ba"
server_key_version: "encryptionkey@0"
I20260806 01:07:13.086720 27133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:13.101485 27133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:13.105690 27133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:13.105993 27133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:13.125535 27133 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33571
I20260806 01:07:13.125572 27234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33571 every 8 connection(s)
I20260806 01:07:13.127010 27133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:13.130874 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:13.131265 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 27133
I20260806 01:07:13.131512 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:13.132252 22781 external_mini_cluster.cc:1428] Setting key 48dee26564c3bc16989963d8bb230a8c
I20260806 01:07:13.145752 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8: Bootstrap starting.
Aug 06 01:07:13 dist-test-slave-7f01 krb5kdc[27115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:13.149662 27236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:13.151070 27236 log.cc:824] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:13.154757 27236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8: No bootstrap required, opened a new log
I20260806 01:07:13.157147 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:13.134105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.22.63.126:33571)
0806 01:07:13.134932 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:13.134944 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:13.134988 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:07:13.136633 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:13.136656 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:13.137050 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:13.137385 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:13.139335 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:13.140658 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:13.142523 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:13.143231 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:13.143368 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:13.148173 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:13.148184 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:13.148202 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:13.148254 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:13.152286 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:13.153071 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:13.153079 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:13.153087 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:13.153290 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:13.153676 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:13.153680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:13.153683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:13.154425 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:13.154706 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:13.155277 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 01:07:13.155775 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":378,"threads_started":1}
I20260806 01:07:13.161371 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:07:13.163981 27236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } }
I20260806 01:07:13.164366 27236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:13.164492 27236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa058eb5bd64a4089c30bb5f61e64a8, State: Initialized, Role: FOLLOWER
I20260806 01:07:13.165057 27236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } }
I20260806 01:07:13.165288 27236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:13.165429 27236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:13.165609 27236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:13.167995 27236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } }
I20260806 01:07:13.168534 27236 leader_election.cc:302] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aa058eb5bd64a4089c30bb5f61e64a8; no voters:
I20260806 01:07:13.169210 27236 leader_election.cc:288] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:13.169389 27247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:13.170657 27247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa058eb5bd64a4089c30bb5f61e64a8, State: Running, Role: LEADER
I20260806 01:07:13.171243 27247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } }
I20260806 01:07:13.171769 27236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:13.174682 27250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aa058eb5bd64a4089c30bb5f61e64a8. Latest consensus state: current_term: 1 leader_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } } }
I20260806 01:07:13.175078 27250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:13.175648 27249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aa058eb5bd64a4089c30bb5f61e64a8" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33571 } } }
I20260806 01:07:13.176136 27249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:13.178226 27257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:13.185009 27257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:13.198371 27270 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:07:13.200412 27257 catalog_manager.cc:1347] Generated new cluster ID: 906a17818a6e41129a97578c22a2c947
I20260806 01:07:13.200547 27257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:13.217309 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:13.209542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.22.63.126:33571)
0806 01:07:13.209780 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:13.209790 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:13.210415 (+ 625us) server_negotiation.cc:407] Connection header received
0806 01:07:13.210629 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:13.210640 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:13.210705 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:13.210815 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:13.211905 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:13.212692 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:13.214007 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:13.214462 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:13.214601 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:13.215788 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:13.215795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:13.215800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:13.216457 (+ 657us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:13.216506 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:13.216529 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:13.216668 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:13.216798 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 01:07:13.216917 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:07:13.226927 27257 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:13.228369 27257 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:13.247475 27257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9aa058eb5bd64a4089c30bb5f61e64a8: Generated new TSK 0
I20260806 01:07:13.248659 27257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:07:13.258803 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27133
I20260806 01:07:13.260088 27230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:13.353264 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 27133
2026-08-06T01:07:13Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2221 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:13 dist-test-slave-7f01 krb5kdc[27283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:13 dist-test-slave-7f01 krb5kdc[27283](info): set up 2 sockets
Aug 06 01:07:13 dist-test-slave-7f01 krb5kdc[27283](info): commencing operation
krb5kdc: starting...
W20260806 01:07:14.659467 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:14 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:14.895905 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33531
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:15.231125 27299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:15.231405 27299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:15.231494 27299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:15.238611 27299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:15.238718 27299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:15.238765 27299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:15.238823 27299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:15.238876 27299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:15.248700 27299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:15.249984 27299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:15.251269 27299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:15.263255 27306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:15.263288 27308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:15.263840 27305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:15.264315 27299 server_base.cc:1034] running on GCE node
I20260806 01:07:15.265121 27299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:15.266498 27299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:15.267716 27299 hybrid_clock.cc:648] HybridClock initialized: now 1785978435267669 us; error 40 us; skew 500 ppm
I20260806 01:07:15.268183 27299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:15 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978435, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:15.271382 27299 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:15.272792 27299 webserver.cc:460] Webserver started at http://127.22.63.126:35141/ using document root <none> and password file <none>
I20260806 01:07:15.273444 27299 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:15.273547 27299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:15.273802 27299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:15.276796 27299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "24a2a93882ae4d2f87dd343d0e0613c0"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "ac7e460ce24ec85846762cfec3b71b2f"
server_key_iv: "d934075b6a4c6c14697c728b4a703cfb"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.277510 27299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "24a2a93882ae4d2f87dd343d0e0613c0"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "ac7e460ce24ec85846762cfec3b71b2f"
server_key_iv: "d934075b6a4c6c14697c728b4a703cfb"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.282606 27299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:07:15.286037 27315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:15.287447 27299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:07:15.287667 27299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "24a2a93882ae4d2f87dd343d0e0613c0"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "ac7e460ce24ec85846762cfec3b71b2f"
server_key_iv: "d934075b6a4c6c14697c728b4a703cfb"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.287916 27299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:15.309530 27299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:15.313534 27299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:15.313830 27299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:15.332657 27299 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33531
I20260806 01:07:15.332700 27393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33531 every 8 connection(s)
I20260806 01:07:15.334033 27299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:15.337953 27394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:15.339542 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 27299
I20260806 01:07:15.339783 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:15.340456 22781 external_mini_cluster.cc:1428] Setting key 86546c26c864e2726c5c06d4e99d3105
I20260806 01:07:15.351322 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0: Bootstrap starting.
I20260806 01:07:15.354797 27394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0: Neither blocks nor log segments found. Creating new log.
Aug 06 01:07:15 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:15.355921 27394 log.cc:824] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:15.359380 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0: No bootstrap required, opened a new log
I20260806 01:07:15.364876 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:15.342592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.22.63.126:33531)
0806 01:07:15.343414 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:15.343429 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:15.343491 (+ 62us) server_negotiation.cc:407] Connection header received
0806 01:07:15.345814 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:15.345845 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:15.346323 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:15.346763 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:15.348072 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.350127 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.351359 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.352379 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.352563 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:15.356600 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:15.356614 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:15.356632 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:15.356693 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:15.360496 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:15.361138 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:15.361151 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:15.361160 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:15.361370 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:15.361730 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:15.361735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:15.361739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:15.362417 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:15.362750 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:15.363038 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 01:07:15.363463 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":344,"threads_started":1}
I20260806 01:07:15.369331 27394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } }
I20260806 01:07:15.369649 27394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:15.369722 27394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a2a93882ae4d2f87dd343d0e0613c0, State: Initialized, Role: FOLLOWER
I20260806 01:07:15.370180 27394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } }
I20260806 01:07:15.370399 27394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:15.370553 27394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:15.370748 27394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:15.372999 27394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } }
I20260806 01:07:15.373517 27394 leader_election.cc:302] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a2a93882ae4d2f87dd343d0e0613c0; no voters:
I20260806 01:07:15.374099 27394 leader_election.cc:288] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:15.374231 27400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:15.374920 27400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [term 1 LEADER]: Becoming Leader. State: Replica: 24a2a93882ae4d2f87dd343d0e0613c0, State: Running, Role: LEADER
I20260806 01:07:15.375556 27400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } }
I20260806 01:07:15.376207 27394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:15.378628 27403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a2a93882ae4d2f87dd343d0e0613c0. Latest consensus state: current_term: 1 leader_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } } }
I20260806 01:07:15.378974 27403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:15.379998 27408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:15.380273 27401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a2a93882ae4d2f87dd343d0e0613c0" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33531 } } }
I20260806 01:07:15.380721 27401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:15.386240 27408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:15.395040 27408 catalog_manager.cc:1347] Generated new cluster ID: acff8778a745427587765c270e147e0d
I20260806 01:07:15.395174 27408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:15.409665 27408 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:15.411594 27408 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:15.418912 27408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a2a93882ae4d2f87dd343d0e0613c0: Generated new TSK 0
I20260806 01:07:15.419960 27408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:07:15.481335 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:15.832787 27426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:15.833069 27426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:15.833230 27426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:15.840504 27426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:15.840617 27426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:15.840811 27426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:15.850605 27426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:15.851977 27426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:15.853358 27426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:15.867478 27432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:15.867666 27433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:15.867687 27435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:15.868363 27426 server_base.cc:1034] running on GCE node
I20260806 01:07:15.869073 27426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:15.870051 27426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:15.871279 27426 hybrid_clock.cc:648] HybridClock initialized: now 1785978435871224 us; error 46 us; skew 500 ppm
I20260806 01:07:15.871652 27426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:15 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978435, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:15.875082 27426 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:07:15.877105 27426 webserver.cc:460] Webserver started at http://127.22.63.65:46725/ using document root <none> and password file <none>
I20260806 01:07:15.878149 27426 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:15.878321 27426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:15.878654 27426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:15.882191 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "57977f615fba4fdea4e8c533f9fae308"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "da207bf74a2ac9cede88b29aad8e11e4"
server_key_iv: "ac2f3bc2145284fc596d64ff58c282b8"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.883260 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "57977f615fba4fdea4e8c533f9fae308"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "da207bf74a2ac9cede88b29aad8e11e4"
server_key_iv: "ac2f3bc2145284fc596d64ff58c282b8"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.889655 27426 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 01:07:15.893390 27445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:15.894819 27426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:07:15.894960 27426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "57977f615fba4fdea4e8c533f9fae308"
format_stamp: "Formatted at 2026-08-06 01:07:15 on dist-test-slave-7f01"
server_key: "da207bf74a2ac9cede88b29aad8e11e4"
server_key_iv: "ac2f3bc2145284fc596d64ff58c282b8"
server_key_version: "encryptionkey@0"
I20260806 01:07:15.895143 27426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:15.913210 27426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:15.917512 27426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:15.917775 27426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:15.918813 27426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:15.920318 27426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:15.920403 27426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:15.920553 27426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:15.920614 27426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:15.955704 27426 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:37437
I20260806 01:07:15.955823 27598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:37437 every 8 connection(s)
I20260806 01:07:15.957178 27426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:15.967406 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 27426
I20260806 01:07:15.967640 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:15.968417 22781 external_mini_cluster.cc:1428] Setting key f00a51dd6000e3e4f4a298b087a43bce
Aug 06 01:07:15 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978435, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:15.978729 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:15.959810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:47203 (local address 127.22.63.126:33531)
0806 01:07:15.960285 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:15.960290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:15.961238 (+ 948us) server_negotiation.cc:407] Connection header received
0806 01:07:15.963096 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:15.963101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:15.963193 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:15.963318 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:15.964865 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.965605 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.967183 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.967616 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.967785 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:15.971391 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:15.971399 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:15.971407 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:15.971451 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:15.975100 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:15.975895 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:15.975905 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:15.975911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:15.976066 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:15.976563 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:15.976568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:15.976574 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:15.977002 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:15.977151 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:15.977944 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 01:07:15.978193 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
I20260806 01:07:15.979395 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:15.960388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33531 (local address 127.22.63.65:47203)
0806 01:07:15.961089 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:15.961127 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:15.962826 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:15.963561 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:15.963569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:15.964136 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:15.964690 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:15.964707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.965777 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.965780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:15.967014 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:15.967026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:15.967874 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:15.967884 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:15.968117 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:15.968815 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:15.968842 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:15.971165 (+ 2323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:15.975262 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:15.975269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:15.975280 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:15.975708 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:15.976180 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:15.976183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:15.976185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:15.976438 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:15.977138 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:15.977144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:15.977524 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 01:07:15.977825 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 01:07:15.978223 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":275,"threads_started":1}
I20260806 01:07:15.981070 27599 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33531
I20260806 01:07:15.981406 27599 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:15.982218 27599 heartbeater.cc:511] Master 127.22.63.126:33531 requested a full tablet report, sending...
I20260806 01:07:15.984617 27338 ts_manager.cc:194] Registered new tserver with Master: 57977f615fba4fdea4e8c533f9fae308 (127.22.63.65:37437)
I20260806 01:07:15.986869 27338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:47203
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:07:16.024772 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:16.359853 27608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:16.360137 27608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:16.360301 27608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:16.367493 27608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:16.367596 27608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:16.367805 27608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:16.377826 27608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:16.379153 27608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:16.380563 27608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:16.394120 27614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:16.394083 27616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:16.394210 27619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:16.397282 27608 server_base.cc:1034] running on GCE node
I20260806 01:07:16.397933 27608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:16.398928 27608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:16.400135 27608 hybrid_clock.cc:648] HybridClock initialized: now 1785978436400098 us; error 32 us; skew 500 ppm
I20260806 01:07:16.400467 27608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:16 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978436, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:16.407593 27608 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:16.409214 27608 webserver.cc:460] Webserver started at http://127.22.63.66:34413/ using document root <none> and password file <none>
I20260806 01:07:16.409983 27608 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:16.410135 27608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:16.410461 27608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:16.413455 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "4b7cbed0bc9a4957a314e03a9d27cb99"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "7f543d7c0a0afc5d1c68a8672641a8ef"
server_key_iv: "0fd385a554e41b99e5563c15d7eea355"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.414251 27608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "4b7cbed0bc9a4957a314e03a9d27cb99"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "7f543d7c0a0afc5d1c68a8672641a8ef"
server_key_iv: "0fd385a554e41b99e5563c15d7eea355"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.419380 27608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:16.422837 27626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:16.424263 27608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:07:16.424455 27608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "4b7cbed0bc9a4957a314e03a9d27cb99"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "7f543d7c0a0afc5d1c68a8672641a8ef"
server_key_iv: "0fd385a554e41b99e5563c15d7eea355"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.424655 27608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:16.436908 27608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:16.441241 27608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:16.441556 27608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:16.442505 27608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:16.443934 27608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:16.444039 27608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:16.444190 27608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:16.444264 27608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:16.478680 27608 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:43497
I20260806 01:07:16.478739 27790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:43497 every 8 connection(s)
I20260806 01:07:16.480337 27608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:16.488161 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 27608
I20260806 01:07:16.488363 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:16.488822 22781 external_mini_cluster.cc:1428] Setting key 557e17562020d6773642824d0c6b82c5
Aug 06 01:07:16 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978436, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:16.501917 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:16.483060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:43499 (local address 127.22.63.126:33531)
0806 01:07:16.483538 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:16.483543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:16.484501 (+ 958us) server_negotiation.cc:407] Connection header received
0806 01:07:16.486312 (+ 1811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:16.486317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:16.486410 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:16.486541 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:16.488115 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:16.488960 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:16.490722 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:16.491223 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:16.491361 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:16.495686 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:16.495691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:16.495695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:16.495727 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:16.498502 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:16.499496 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:16.499500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:16.499502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:16.499631 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:16.500189 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:16.500193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:16.500195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:16.500474 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:16.500588 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:16.501489 (+ 901us) server_negotiation.cc:299] Negotiation successful
0806 01:07:16.501654 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":243,"threads_started":1}
I20260806 01:07:16.503520 27794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:16.483579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33531 (local address 127.22.63.66:43499)
0806 01:07:16.484335 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:16.484380 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:16.486047 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:16.486787 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:16.486794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:16.487296 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:16.487945 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:16.487964 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:16.489130 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:16.489134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:16.490575 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:16.490593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:16.491449 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:16.491458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:16.491723 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:16.492584 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:16.492607 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:16.495443 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:16.498644 (+ 3201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:16.498653 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:16.498670 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:16.499328 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:16.499762 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:16.499770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:16.499774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:16.500045 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:16.500610 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:16.500619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:16.501093 (+ 474us) client_negotiation.cc:769] Sending connection context
0806 01:07:16.501395 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 01:07:16.502000 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":337,"threads_started":1}
I20260806 01:07:16.505064 27791 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33531
I20260806 01:07:16.505435 27791 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:16.506377 27791 heartbeater.cc:511] Master 127.22.63.126:33531 requested a full tablet report, sending...
I20260806 01:07:16.508037 27338 ts_manager.cc:194] Registered new tserver with Master: 4b7cbed0bc9a4957a314e03a9d27cb99 (127.22.63.66:43497)
I20260806 01:07:16.508869 27338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:43499
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:07:16.543955 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:16.894142 27799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:16.894608 27799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:16.894804 27799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:16.908777 27799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:16.908977 27799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:16.909216 27799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:16.921736 27799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:16.923152 27799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:16.924595 27799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:16.938683 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:16.938731 27807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:16.938767 27805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:16.940094 27799 server_base.cc:1034] running on GCE node
I20260806 01:07:16.940752 27799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:16.941754 27799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:16.942996 27799 hybrid_clock.cc:648] HybridClock initialized: now 1785978436942938 us; error 45 us; skew 500 ppm
I20260806 01:07:16.943332 27799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:16 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978436, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:16.946744 27799 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:16.948395 27799 webserver.cc:460] Webserver started at http://127.22.63.67:46215/ using document root <none> and password file <none>
I20260806 01:07:16.949115 27799 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:16.949268 27799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:16.949607 27799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:16.952639 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "ba590816e0b9467ca657945654f7e44a"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "b457d207121e453e34c766472629278a"
server_key_iv: "696d42f0f0cb684612cd1b855ad4234f"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.953446 27799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "ba590816e0b9467ca657945654f7e44a"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "b457d207121e453e34c766472629278a"
server_key_iv: "696d42f0f0cb684612cd1b855ad4234f"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.958851 27799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:07:16.962440 27819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:16.963882 27799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:07:16.964061 27799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "ba590816e0b9467ca657945654f7e44a"
format_stamp: "Formatted at 2026-08-06 01:07:16 on dist-test-slave-7f01"
server_key: "b457d207121e453e34c766472629278a"
server_key_iv: "696d42f0f0cb684612cd1b855ad4234f"
server_key_version: "encryptionkey@0"
I20260806 01:07:16.964269 27799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:16.987078 27799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:16.990052 27599 heartbeater.cc:503] Master 127.22.63.126:33531 was elected leader, sending a full tablet report...
I20260806 01:07:16.991920 27799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:16.992283 27799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:16.993330 27799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:16.994601 27799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:16.994678 27799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:16.994761 27799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:16.994805 27799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:17.029104 27799 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:39215
I20260806 01:07:17.029198 27974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:39215 every 8 connection(s)
I20260806 01:07:17.030419 27799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:17.031939 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 27799
I20260806 01:07:17.032127 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:17.032712 22781 external_mini_cluster.cc:1428] Setting key 9e7df82d38346f141eed4c6d0c030da0
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978436, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:17.050873 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.033302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:45739 (local address 127.22.63.126:33531)
0806 01:07:17.033876 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:17.033882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:17.034931 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 01:07:17.036917 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:17.036923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:17.037032 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:17.037171 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:17.038612 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.039403 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.041000 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.041402 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.041551 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.045072 (+ 3521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:17.045077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:17.045080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:17.045112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:17.048142 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.048891 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.048895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.048897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.049018 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.049389 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.049392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.049394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.049660 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:17.049775 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:17.050394 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 01:07:17.050579 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":275,"threads_started":1}
I20260806 01:07:17.051833 27979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.033900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33531 (local address 127.22.63.67:45739)
0806 01:07:17.034777 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:17.034816 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:17.036664 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:17.037397 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:17.037405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:17.037890 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:17.038439 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:17.038455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.039573 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.039577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:17.040834 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:17.040846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.041640 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.041648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:17.041887 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.042607 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:17.042630 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:17.044845 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:17.048269 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:17.048276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:17.048288 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:17.048770 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:17.049113 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:17.049116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:17.049118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:17.049292 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:17.049765 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:17.049771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:17.050072 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 01:07:17.050300 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 01:07:17.050685 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":337,"threads_started":1}
I20260806 01:07:17.053150 27975 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33531
I20260806 01:07:17.053467 27975 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:17.054215 27975 heartbeater.cc:511] Master 127.22.63.126:33531 requested a full tablet report, sending...
I20260806 01:07:17.055583 27338 ts_manager.cc:194] Registered new tserver with Master: ba590816e0b9467ca657945654f7e44a (127.22.63.67:39215)
I20260806 01:07:17.056399 27338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:45739
I20260806 01:07:17.060679 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:17.117492 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.103311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.22.63.126:33531)
0806 01:07:17.103586 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:17.103597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:17.104038 (+ 441us) server_negotiation.cc:407] Connection header received
0806 01:07:17.104187 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:17.104191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:17.104261 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:17.104347 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:17.105730 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.106578 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.107956 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.108364 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.108483 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.112293 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:17.112298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:17.112301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:17.112337 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:17.115006 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.115609 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.115613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.115615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.115785 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.116179 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.116182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.116184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.116514 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:17.116695 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:17.116949 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 01:07:17.117130 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 01:07:17.118064 27338 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42772
I20260806 01:07:17.119946 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27426
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:17.135782 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.121067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.22.63.65:37437)
0806 01:07:17.121692 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:17.121700 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:17.121723 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:07:17.121780 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:17.121786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:17.121987 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:17.122110 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:17.123254 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.124414 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.125560 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.126129 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.126239 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.130308 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:17.130319 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:17.130334 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:17.130370 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:17.133234 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.133849 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.133856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.133858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.133978 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.134300 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.134303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.134305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.134648 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:17.134874 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:17.135109 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 01:07:17.135305 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":339,"threads_started":1}
W20260806 01:07:17.137202 27594 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37384
W20260806 01:07:17.138329 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27426: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:17.138561 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 27426
I20260806 01:07:17.164912 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27608
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:07:17.181358 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.165877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.22.63.66:43497)
0806 01:07:17.166573 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:17.166580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:17.166607 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:07:17.166666 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:17.166672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:17.166899 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:17.167033 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:17.168464 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.169643 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.170992 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.171547 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.171654 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.175819 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:17.175838 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:17.175852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:17.175888 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:17.178785 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.179426 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.179433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.179435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.179566 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.180033 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.180036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.180039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.180373 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:17.180619 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:17.180870 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 01:07:17.181070 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":398,"threads_started":1}
W20260806 01:07:17.182704 27784 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36530
W20260806 01:07:17.183949 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27608: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:17.184113 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 27608
I20260806 01:07:17.209216 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27799
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[27283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:07:17.226168 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:17.210191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.22.63.67:39215)
0806 01:07:17.210872 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:17.210880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:17.210913 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:07:17.210981 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:17.210987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:17.211203 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:17.211346 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:17.212791 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.213894 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.215174 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:17.215872 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:17.215982 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:17.220341 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:17.220354 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:17.220368 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:17.220413 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:17.223381 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.224214 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.224221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.224223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.224347 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:17.224706 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:17.224709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:17.224711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:17.225103 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:17.225343 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:17.225632 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 01:07:17.225871 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":335,"threads_started":1}
W20260806 01:07:17.227480 27968 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36660
W20260806 01:07:17.228735 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27799: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:07:17.228933 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 27799
I20260806 01:07:17.254292 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 27299
I20260806 01:07:17.255584 27387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:17.361261 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 27299
2026-08-06T01:07:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4010 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[28010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[28010](info): set up 2 sockets
Aug 06 01:07:17 dist-test-slave-7f01 krb5kdc[28010](info): commencing operation
krb5kdc: starting...
W20260806 01:07:18.672360 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.003s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:18 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:18.915177 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33575
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33575
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:19.248172 28027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:19.248468 28027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:19.248688 28027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:19.255831 28027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:19.255941 28027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:19.256057 28027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:19.256140 28027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:19.256232 28027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:19.266032 28027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:19.267475 28027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:19.268844 28027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:19.280684 28037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:19.280750 28034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:19.280754 28035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:19.282028 28027 server_base.cc:1034] running on GCE node
I20260806 01:07:19.282821 28027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:19.284327 28027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:19.285586 28027 hybrid_clock.cc:648] HybridClock initialized: now 1785978439285535 us; error 39 us; skew 500 ppm
I20260806 01:07:19.285981 28027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:19 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978439, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:19.289302 28027 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:19.290913 28027 webserver.cc:460] Webserver started at http://127.22.63.126:46339/ using document root <none> and password file <none>
I20260806 01:07:19.291630 28027 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:19.291816 28027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:19.292141 28027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:19.295113 28027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "e8d8242fe4404efa99a175daab221a21"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "9d931799d64878aecdffee5bb15c92ba"
server_key_iv: "95890d287eff1054f68139ac5e092cc3"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.296006 28027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "e8d8242fe4404efa99a175daab221a21"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "9d931799d64878aecdffee5bb15c92ba"
server_key_iv: "95890d287eff1054f68139ac5e092cc3"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.301116 28027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:07:19.304606 28046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:19.305871 28027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:19.306061 28027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "e8d8242fe4404efa99a175daab221a21"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "9d931799d64878aecdffee5bb15c92ba"
server_key_iv: "95890d287eff1054f68139ac5e092cc3"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.306253 28027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:19.328058 28027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:19.332581 28027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:19.332883 28027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:19.351845 28027 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33575
I20260806 01:07:19.351907 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33575 every 8 connection(s)
I20260806 01:07:19.353240 28027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:19.357061 28125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:19.358490 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28027
I20260806 01:07:19.358742 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:19.359663 22781 external_mini_cluster.cc:1428] Setting key b7b93db3fc625284e7d5c4719b76b890
I20260806 01:07:19.371551 28125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21: Bootstrap starting.
Aug 06 01:07:19 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:19.375604 28125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:19.376739 28125 log.cc:824] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:19.379911 28125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21: No bootstrap required, opened a new log
I20260806 01:07:19.383477 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:19.361864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48802 (local address 127.22.63.126:33575)
0806 01:07:19.362699 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:19.362712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:19.362753 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:07:19.364438 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:19.364464 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:19.364857 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:19.365170 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:19.367095 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:19.368592 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:19.370413 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:19.371115 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:19.371258 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:19.376111 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:19.376118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:19.376134 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:19.376178 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:19.379307 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:19.380097 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:19.380105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:19.380110 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:19.380275 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:19.380674 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:19.380679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:19.380683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:19.381203 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:19.381550 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:19.382027 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:07:19.382449 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":389,"threads_started":1}
I20260806 01:07:19.388041 28125 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } }
I20260806 01:07:19.388406 28125 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:19.388515 28125 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d8242fe4404efa99a175daab221a21, State: Initialized, Role: FOLLOWER
I20260806 01:07:19.389109 28125 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } }
I20260806 01:07:19.389366 28125 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:19.389523 28125 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:19.389703 28125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:19.392166 28125 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } }
I20260806 01:07:19.392761 28125 leader_election.cc:302] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8d8242fe4404efa99a175daab221a21; no voters:
I20260806 01:07:19.393381 28125 leader_election.cc:288] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:19.393535 28130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:19.394240 28130 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [term 1 LEADER]: Becoming Leader. State: Replica: e8d8242fe4404efa99a175daab221a21, State: Running, Role: LEADER
I20260806 01:07:19.394804 28130 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } }
I20260806 01:07:19.395540 28125 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:19.398878 28132 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8d8242fe4404efa99a175daab221a21. Latest consensus state: current_term: 1 leader_uuid: "e8d8242fe4404efa99a175daab221a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } } }
I20260806 01:07:19.398854 28131 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8d8242fe4404efa99a175daab221a21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8d8242fe4404efa99a175daab221a21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33575 } } }
I20260806 01:07:19.399336 28132 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:19.399336 28131 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:19.409108 28136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:19.416363 28136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:19.425743 28136 catalog_manager.cc:1347] Generated new cluster ID: 9c5478fa2abc4040a753fbfb6802359a
I20260806 01:07:19.425912 28136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:19.433784 28136 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:19.435176 28136 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:19.442510 28136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8d8242fe4404efa99a175daab221a21: Generated new TSK 0
I20260806 01:07:19.443572 28136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:07:19.507419 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:19.854152 28158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:19.854678 28158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:19.854875 28158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:19.862475 28158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:19.862741 28158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:19.862962 28158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:19.876042 28158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:19.877868 28158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:19.879395 28158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:19.895006 28164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:19.895061 28169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:19.895012 28165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:19.896399 28158 server_base.cc:1034] running on GCE node
I20260806 01:07:19.897541 28158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:19.898607 28158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:19.899849 28158 hybrid_clock.cc:648] HybridClock initialized: now 1785978439899806 us; error 28 us; skew 500 ppm
I20260806 01:07:19.900266 28158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:19 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978439, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:19.904033 28158 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:07:19.905642 28158 webserver.cc:460] Webserver started at http://127.22.63.65:41849/ using document root <none> and password file <none>
I20260806 01:07:19.906427 28158 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:19.906605 28158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:19.906983 28158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:19.910670 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "7748abd4a35b479b9f28178bf9a082cf"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "8de6896a9d29bcb32f922c0fd1110df1"
server_key_iv: "512f27e76a025f99d028c493fb170297"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.911643 28158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "7748abd4a35b479b9f28178bf9a082cf"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "8de6896a9d29bcb32f922c0fd1110df1"
server_key_iv: "512f27e76a025f99d028c493fb170297"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.917733 28158 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 01:07:19.921792 28179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:19.923254 28158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:19.923542 28158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "7748abd4a35b479b9f28178bf9a082cf"
format_stamp: "Formatted at 2026-08-06 01:07:19 on dist-test-slave-7f01"
server_key: "8de6896a9d29bcb32f922c0fd1110df1"
server_key_iv: "512f27e76a025f99d028c493fb170297"
server_key_version: "encryptionkey@0"
I20260806 01:07:19.923842 28158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:19.953044 28158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:19.957696 28158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:19.958033 28158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:19.959044 28158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:19.960477 28158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:19.960582 28158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:19.960738 28158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:19.960814 28158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:19.996281 28158 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39087
I20260806 01:07:19.996317 28347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39087 every 8 connection(s)
I20260806 01:07:19.997642 28158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:20.002465 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28158
I20260806 01:07:20.002703 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:20.003345 22781 external_mini_cluster.cc:1428] Setting key a7cca340b703969905b80625fb3b27db
Aug 06 01:07:20 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978439, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:20.020747 28350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:20.000213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54631 (local address 127.22.63.126:33575)
0806 01:07:20.000688 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:20.000692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:20.001560 (+ 868us) server_negotiation.cc:407] Connection header received
0806 01:07:20.003597 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:20.003602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:20.003696 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:20.003845 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:20.005771 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.006485 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.009091 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.009639 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.009772 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:20.014369 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:20.014377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:20.014385 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:20.014433 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:20.017316 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:20.018292 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:20.018296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:20.018299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:20.018421 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:20.018940 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:20.018944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:20.018946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:20.019217 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:20.019332 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:20.020288 (+ 956us) server_negotiation.cc:299] Negotiation successful
0806 01:07:20.020465 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":262,"threads_started":1}
I20260806 01:07:20.022404 28351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:20.000657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33575 (local address 127.22.63.65:54631)
0806 01:07:20.001396 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:20.001436 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:20.003289 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:20.004116 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:20.004124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:20.004789 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:20.005588 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:20.005614 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.006663 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.006672 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:20.008916 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:20.008946 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.009870 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.009880 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:20.010235 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:20.011089 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:20.011124 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:20.014132 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:20.017467 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:20.017479 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:20.017499 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:20.018162 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:20.018538 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:20.018545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:20.018552 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:20.018814 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:20.019365 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:20.019377 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:20.019878 (+ 501us) client_negotiation.cc:769] Sending connection context
0806 01:07:20.020207 (+ 329us) client_negotiation.cc:240] Negotiation successful
0806 01:07:20.020811 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":314,"threads_started":1}
I20260806 01:07:20.024009 28348 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33575
I20260806 01:07:20.024362 28348 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:20.025341 28348 heartbeater.cc:511] Master 127.22.63.126:33575 requested a full tablet report, sending...
I20260806 01:07:20.028087 28069 ts_manager.cc:194] Registered new tserver with Master: 7748abd4a35b479b9f28178bf9a082cf (127.22.63.65:39087)
I20260806 01:07:20.030366 28069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:54631
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:07:20.062578 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:20.418352 28358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:20.418676 28358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:20.418835 28358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:20.425921 28358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:20.426069 28358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:20.426265 28358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:20.435827 28358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:20.437240 28358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:20.438642 28358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:20.452747 28364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:20.452832 28371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:20.452790 28368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:20.453617 28358 server_base.cc:1034] running on GCE node
I20260806 01:07:20.454178 28358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:20.455087 28358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:20.456287 28358 hybrid_clock.cc:648] HybridClock initialized: now 1785978440456223 us; error 52 us; skew 500 ppm
I20260806 01:07:20.456570 28358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:20 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978440, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:20.459792 28358 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:20.461225 28358 webserver.cc:460] Webserver started at http://127.22.63.66:45987/ using document root <none> and password file <none>
I20260806 01:07:20.461896 28358 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:20.462003 28358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:20.462287 28358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:20.465350 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "b172440a9ef947d38b6e6d4a34842c1e"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "904a0774ba8302fa0d2b10eae2f85cba"
server_key_iv: "5fd18049e53feede12d4f0a9874731e2"
server_key_version: "encryptionkey@0"
I20260806 01:07:20.466118 28358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "b172440a9ef947d38b6e6d4a34842c1e"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "904a0774ba8302fa0d2b10eae2f85cba"
server_key_iv: "5fd18049e53feede12d4f0a9874731e2"
server_key_version: "encryptionkey@0"
I20260806 01:07:20.471401 28358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260806 01:07:20.475200 28379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:20.476729 28358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:20.476874 28358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "b172440a9ef947d38b6e6d4a34842c1e"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "904a0774ba8302fa0d2b10eae2f85cba"
server_key_iv: "5fd18049e53feede12d4f0a9874731e2"
server_key_version: "encryptionkey@0"
I20260806 01:07:20.477072 28358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:20.501314 28358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:20.505579 28358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:20.505841 28358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:20.506829 28358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:20.508334 28358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:20.508407 28358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:20.508498 28358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:20.508544 28358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:20.543231 28358 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:34961
I20260806 01:07:20.543318 28535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:34961 every 8 connection(s)
I20260806 01:07:20.544698 28358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:20.547194 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28358
I20260806 01:07:20.547413 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:20.548048 22781 external_mini_cluster.cc:1428] Setting key ba602d5e90a928d027013ac0c8d27690
Aug 06 01:07:20 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978440, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:20.569036 28538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:20.547479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:37437 (local address 127.22.63.126:33575)
0806 01:07:20.548034 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:20.548041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:20.549205 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 01:07:20.551631 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:20.551635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:20.551700 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:20.551857 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:20.554076 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.555142 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.557676 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.558373 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.558497 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:20.562846 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:20.562851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:20.562854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:20.562890 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:20.565658 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:20.566480 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:20.566486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:20.566490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:20.566623 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:20.567107 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:20.567112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:20.567117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:20.567411 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:20.567576 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:20.568404 (+ 828us) server_negotiation.cc:299] Negotiation successful
0806 01:07:20.568655 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":269,"threads_started":1}
I20260806 01:07:20.570087 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:20.548038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33575 (local address 127.22.63.66:37437)
0806 01:07:20.549012 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:20.549067 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:20.551339 (+ 2272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:20.552127 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:20.552143 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:20.553019 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:20.553864 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:20.553899 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.555317 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.555326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:20.557512 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:20.557531 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:20.558617 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:20.558627 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:20.558975 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:20.559895 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:20.559938 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:20.562561 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:20.565838 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:20.565848 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:20.565866 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:20.566336 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:20.566742 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:20.566747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:20.566751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:20.566980 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:20.567556 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:20.567565 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:20.568011 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 01:07:20.568303 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 01:07:20.568904 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":335,"threads_started":1}
I20260806 01:07:20.571961 28536 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33575
I20260806 01:07:20.572293 28536 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
I20260806 01:07:20.573091 28536 heartbeater.cc:511] Master 127.22.63.126:33575 requested a full tablet report, sending...
I20260806 01:07:20.574723 28069 ts_manager.cc:194] Registered new tserver with Master: b172440a9ef947d38b6e6d4a34842c1e (127.22.63.66:34961)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
I20260806 01:07:20.575695 28069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:37437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:07:20.604447 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:20.945322 28543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:20.945631 28543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:20.945814 28543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:20.953292 28543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:20.953449 28543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:20.953655 28543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:20.963553 28543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33575
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:20.964960 28543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:20.966353 28543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:20.979938 28554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:20.980064 28551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:20.980038 28552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:20.981271 28543 server_base.cc:1034] running on GCE node
I20260806 01:07:20.981973 28543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:20.982959 28543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:20.984200 28543 hybrid_clock.cc:648] HybridClock initialized: now 1785978440984168 us; error 51 us; skew 500 ppm
I20260806 01:07:20.984529 28543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:20 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978440, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:20.987885 28543 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:20.989463 28543 webserver.cc:460] Webserver started at http://127.22.63.67:36669/ using document root <none> and password file <none>
I20260806 01:07:20.990180 28543 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:20.990329 28543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:20.990658 28543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:20.993805 28543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "d571a6391ce14022a50f06d33819f597"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "48224e38a03873b645793578406bf2b6"
server_key_iv: "6fc344749a9cd1ccac3e6cba123d9c47"
server_key_version: "encryptionkey@0"
I20260806 01:07:20.994625 28543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "d571a6391ce14022a50f06d33819f597"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "48224e38a03873b645793578406bf2b6"
server_key_iv: "6fc344749a9cd1ccac3e6cba123d9c47"
server_key_version: "encryptionkey@0"
I20260806 01:07:21.000520 28543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:07:21.003931 28565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:21.005270 28543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:07:21.005432 28543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "d571a6391ce14022a50f06d33819f597"
format_stamp: "Formatted at 2026-08-06 01:07:20 on dist-test-slave-7f01"
server_key: "48224e38a03873b645793578406bf2b6"
server_key_iv: "6fc344749a9cd1ccac3e6cba123d9c47"
server_key_version: "encryptionkey@0"
I20260806 01:07:21.005652 28543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:21.029320 28543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:21.033718 28543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:21.033780 28348 heartbeater.cc:503] Master 127.22.63.126:33575 was elected leader, sending a full tablet report...
I20260806 01:07:21.034072 28543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:21.035322 28543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:21.036942 28543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:21.037031 28543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:21.037175 28543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:21.037242 28543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:21.070480 28543 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:40569
I20260806 01:07:21.070544 28733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:40569 every 8 connection(s)
I20260806 01:07:21.072016 28543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:21.078440 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28543
I20260806 01:07:21.078662 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:21.079281 22781 external_mini_cluster.cc:1428] Setting key 620864128a12599c6f531f526a41d89c
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978440, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:21.092420 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.074613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:40119 (local address 127.22.63.126:33575)
0806 01:07:21.075061 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.075066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.076103 (+ 1037us) server_negotiation.cc:407] Connection header received
0806 01:07:21.077950 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.077955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.078051 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.078177 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.079875 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.081020 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.082712 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.083117 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.083255 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.086663 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.086668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.086671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.086703 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.089346 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.090082 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.090092 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.090094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.090214 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.090659 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.090668 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.090670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.090925 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.091095 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.091805 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.092029 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":209,"threads_started":1}
I20260806 01:07:21.093598 28737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.075174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33575 (local address 127.22.63.67:40119)
0806 01:07:21.075943 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:21.075988 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:21.077681 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:21.078405 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:21.078412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:21.079041 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:21.079661 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:21.079687 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.081186 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.081190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:21.082558 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:21.082570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.083348 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.083357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:21.083588 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.084261 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:21.084283 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:21.086443 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:21.089500 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:21.089509 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:21.089524 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:21.089950 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:21.090330 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:21.090336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:21.090339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:21.090532 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:21.091064 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:21.091072 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:21.091415 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 01:07:21.091684 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 01:07:21.092228 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":278,"threads_started":1}
I20260806 01:07:21.094993 28734 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33575
I20260806 01:07:21.095259 28734 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:21.096060 28734 heartbeater.cc:511] Master 127.22.63.126:33575 requested a full tablet report, sending...
I20260806 01:07:21.097551 28069 ts_manager.cc:194] Registered new tserver with Master: d571a6391ce14022a50f06d33819f597 (127.22.63.67:40569)
I20260806 01:07:21.098435 28069 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:40119
I20260806 01:07:21.107259 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:21.165390 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.149556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.22.63.65:39087)
0806 01:07:21.150288 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.150295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.150322 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:07:21.150383 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.150390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.150625 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.150758 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.152352 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.153400 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.154676 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.155258 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.155366 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.159398 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.159409 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.159423 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.159462 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.162455 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.163131 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.163138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.163140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.163265 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.163664 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.163670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.163673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.164086 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.164418 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.164727 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.165025 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":401,"threads_started":1}
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:21.184356 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.170098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.22.63.126:33575)
0806 01:07:21.170355 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.170361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.170530 (+ 169us) server_negotiation.cc:407] Connection header received
0806 01:07:21.170647 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.170652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.170717 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.170796 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.172197 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.173500 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.174728 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.175128 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.175227 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.179200 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.179206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.179209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.179246 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.181941 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.182566 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.182570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.182572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.182698 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.183024 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.183028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.183030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.183350 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.183541 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.183857 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.184034 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260806 01:07:21.184971 28069 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48818
I20260806 01:07:21.185976 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28158
I20260806 01:07:21.199411 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.187022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.22.63.65:39087)
0806 01:07:21.187246 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.187252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.187413 (+ 161us) server_negotiation.cc:407] Connection header received
0806 01:07:21.187611 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.187616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.187724 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.187865 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.189083 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.189784 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.191001 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.191438 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.191583 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.193978 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.193984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.193989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.194034 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.196739 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.197460 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.197465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.197469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.197612 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.197994 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.197999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.198003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.198266 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.198540 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.198852 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.199086 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:07:21.200193 28342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:21.285573 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28158
I20260806 01:07:21.314152 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28358
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:07:21.331657 28749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.315190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.22.63.66:34961)
0806 01:07:21.315841 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.315848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.315884 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:07:21.315947 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.315954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.316171 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.316305 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.317506 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.318536 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.319931 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.320518 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.320628 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.324971 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.324984 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.325000 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.325036 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.328251 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.329346 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.329353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.329355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.329556 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.329953 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.329959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.329963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.330348 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.330677 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.330998 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.331284 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":301,"threads_started":1}
I20260806 01:07:21.332986 28530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:21.416702 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28358
I20260806 01:07:21.440178 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28543
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:07:21.461691 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:21.441367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.22.63.67:40569)
0806 01:07:21.442012 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:21.442018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:21.442048 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:07:21.442106 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:21.442112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:21.442321 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:21.442498 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:21.443870 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.444907 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.446202 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:21.446780 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:21.446922 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:21.455942 (+ 9020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:21.455958 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:21.455981 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:21.456025 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:21.458696 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.459433 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.459440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.459443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.459569 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:21.460009 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:21.460019 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:21.460023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:21.460376 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:21.460701 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:21.461034 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 01:07:21.461336 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":320,"threads_started":1}
I20260806 01:07:21.463049 28729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:21.546316 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28543
I20260806 01:07:21.571491 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28027
I20260806 01:07:21.572855 28119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:21.681857 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28027
2026-08-06T01:07:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4325 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28762](info): set up 2 sockets
Aug 06 01:07:21 dist-test-slave-7f01 krb5kdc[28762](info): commencing operation
krb5kdc: starting...
W20260806 01:07:22.984800 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:23 dist-test-slave-7f01 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:23.223675 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33117
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:23.554939 28778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:23.555222 28778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:23.555387 28778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:23.562568 28778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:23.562680 28778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:23.562763 28778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:23.562848 28778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:23.562960 28778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:23.573038 28778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:23.574357 28778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:23.575721 28778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:23.587685 28787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:23.587954 28786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:23.587937 28789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:23.588974 28778 server_base.cc:1034] running on GCE node
I20260806 01:07:23.589828 28778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:23.591246 28778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:23.592473 28778 hybrid_clock.cc:648] HybridClock initialized: now 1785978443592419 us; error 57 us; skew 500 ppm
I20260806 01:07:23.592898 28778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:23 dist-test-slave-7f01 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:23.596220 28778 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:23.597352 28778 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:23.597455 28778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:23.597743 28778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:23.600880 28778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "65654f01aaff4f8da8b50c6c6f98ba6d"
format_stamp: "Formatted at 2026-08-06 01:07:23 on dist-test-slave-7f01"
server_key: "6e7f176de23acc05c9b4558f06570d1e"
server_key_iv: "596e1db45b212ccf755f328be11faa38"
server_key_version: "encryptionkey@0"
I20260806 01:07:23.601661 28778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "65654f01aaff4f8da8b50c6c6f98ba6d"
format_stamp: "Formatted at 2026-08-06 01:07:23 on dist-test-slave-7f01"
server_key: "6e7f176de23acc05c9b4558f06570d1e"
server_key_iv: "596e1db45b212ccf755f328be11faa38"
server_key_version: "encryptionkey@0"
I20260806 01:07:23.606845 28778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:23.610387 28798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:23.611884 28778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:23.612068 28778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "65654f01aaff4f8da8b50c6c6f98ba6d"
format_stamp: "Formatted at 2026-08-06 01:07:23 on dist-test-slave-7f01"
server_key: "6e7f176de23acc05c9b4558f06570d1e"
server_key_iv: "596e1db45b212ccf755f328be11faa38"
server_key_version: "encryptionkey@0"
I20260806 01:07:23.612313 28778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:23.624385 28778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:23.628584 28778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:23.628890 28778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:23.648080 28778 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33117
I20260806 01:07:23.648101 28868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33117 every 8 connection(s)
I20260806 01:07:23.649055 28778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:23.652761 28869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:23.657486 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28778
I20260806 01:07:23.657742 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:23.658391 22781 external_mini_cluster.cc:1428] Setting key 44553d47c810e62fe39e7fa52c7d2734
I20260806 01:07:23.663834 28869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: Bootstrap starting.
I20260806 01:07:23.667582 28869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:23.668895 28869 log.cc:824] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:23.672510 28869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: No bootstrap required, opened a new log
Aug 06 01:07:23 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:23.681195 28869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } }
I20260806 01:07:23.681738 28869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:23.681860 28869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65654f01aaff4f8da8b50c6c6f98ba6d, State: Initialized, Role: FOLLOWER
I20260806 01:07:23.682555 28869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } }
I20260806 01:07:23.682788 28869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:23.682945 28869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:23.683138 28869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:23.683486 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:23.660437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.22.63.126:33117)
0806 01:07:23.661295 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:23.661307 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:23.661363 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:07:23.663069 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:23.663091 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:23.663462 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:23.663855 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:23.665765 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:23.667127 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:23.669106 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:23.670218 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:23.670390 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:23.675245 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:23.675252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:23.675269 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:23.675307 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:23.678613 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:23.679400 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:23.679411 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:23.679422 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:23.679676 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:23.680254 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:23.680261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:23.680266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:23.680981 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:23.681203 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:23.681758 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 01:07:23.682221 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":410,"threads_started":1}
I20260806 01:07:23.686203 28869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } }
I20260806 01:07:23.686714 28869 leader_election.cc:302] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65654f01aaff4f8da8b50c6c6f98ba6d; no voters:
I20260806 01:07:23.687343 28869 leader_election.cc:288] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:23.687481 28879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:23.688246 28879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [term 1 LEADER]: Becoming Leader. State: Replica: 65654f01aaff4f8da8b50c6c6f98ba6d, State: Running, Role: LEADER
I20260806 01:07:23.688747 28879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } }
I20260806 01:07:23.689510 28869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:23.691329 28880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } } }
I20260806 01:07:23.691658 28880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:23.693011 28881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65654f01aaff4f8da8b50c6c6f98ba6d. Latest consensus state: current_term: 1 leader_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65654f01aaff4f8da8b50c6c6f98ba6d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33117 } } }
I20260806 01:07:23.693388 28881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:23.697263 28887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:23.702634 28887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:23.711671 28887 catalog_manager.cc:1347] Generated new cluster ID: dad43a0d8a284912b70f59b56826d89c
I20260806 01:07:23.711808 28887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:23.725322 28887 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:23.727232 28887 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:23.736184 28887 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: Generated new TSK 0
I20260806 01:07:23.737020 28887 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:07:23.802995 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:24.154496 28907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:24.154775 28907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:24.154955 28907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:24.162159 28907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:24.162263 28907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:24.162465 28907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:24.172148 28907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:24.173450 28907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:24.174780 28907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:24.188218 28912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:24.188267 28917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:24.188313 28914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:24.189142 28907 server_base.cc:1034] running on GCE node
I20260806 01:07:24.189852 28907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:24.190955 28907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:24.192160 28907 hybrid_clock.cc:648] HybridClock initialized: now 1785978444192115 us; error 31 us; skew 500 ppm
I20260806 01:07:24.192495 28907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:24 dist-test-slave-7f01 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978444, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:24.195585 28907 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:07:24.196798 28907 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:24.196935 28907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:24.197278 28907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:24.200486 28907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "1c55820ff11245ecbef4de640c833d1f"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "cbc57d47c04e11ae630cd76ce964a5fa"
server_key_iv: "87f842b431f019ae52a9ef40385bb929"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.201305 28907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "1c55820ff11245ecbef4de640c833d1f"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "cbc57d47c04e11ae630cd76ce964a5fa"
server_key_iv: "87f842b431f019ae52a9ef40385bb929"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.206535 28907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:07:24.210072 28926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.211391 28907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:07:24.211575 28907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "1c55820ff11245ecbef4de640c833d1f"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "cbc57d47c04e11ae630cd76ce964a5fa"
server_key_iv: "87f842b431f019ae52a9ef40385bb929"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.211822 28907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:24.233781 28907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:24.238540 28907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:24.238826 28907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:24.239863 28907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:24.241293 28907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:24.241441 28907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.241566 28907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:24.241657 28907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.276891 28907 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:37737
I20260806 01:07:24.276985 29086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:37737 every 8 connection(s)
I20260806 01:07:24.277829 28907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:24.287858 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 28907
I20260806 01:07:24.288095 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:24.288734 22781 external_mini_cluster.cc:1428] Setting key e1ef576dea643b844926fd46c34e8fd0
Aug 06 01:07:24 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978444, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:24.299140 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:24.280467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:40701 (local address 127.22.63.126:33117)
0806 01:07:24.280940 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:24.280944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:24.281868 (+ 924us) server_negotiation.cc:407] Connection header received
0806 01:07:24.283654 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:24.283659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:24.283734 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:24.283886 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:24.285390 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.286184 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.287861 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.288392 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.288533 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:24.292944 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:24.292952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:24.292957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:24.292997 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:24.295690 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:24.296554 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:24.296561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:24.296565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:24.296697 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:24.297145 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:24.297152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:24.297156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:24.297441 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:24.297603 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:24.298600 (+ 997us) server_negotiation.cc:299] Negotiation successful
0806 01:07:24.298806 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":225,"threads_started":1}
I20260806 01:07:24.300446 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:24.281010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33117 (local address 127.22.63.65:40701)
0806 01:07:24.281700 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:24.281738 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:24.283407 (+ 1669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:24.284077 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:24.284085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:24.284608 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:24.285208 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:24.285226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.286347 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.286360 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:24.287645 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:24.287659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.288644 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.288652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:24.288930 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:24.289732 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:24.289755 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:24.292634 (+ 2879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:24.295923 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:24.295933 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:24.295950 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:24.296418 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:24.296821 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:24.296826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:24.296831 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:24.297026 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:24.297623 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:24.297635 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:24.298097 (+ 462us) client_negotiation.cc:769] Sending connection context
0806 01:07:24.298453 (+ 356us) client_negotiation.cc:240] Negotiation successful
0806 01:07:24.299076 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":278,"threads_started":1}
I20260806 01:07:24.302150 29087 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33117
I20260806 01:07:24.302489 29087 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:24.303267 29087 heartbeater.cc:511] Master 127.22.63.126:33117 requested a full tablet report, sending...
I20260806 01:07:24.305572 28822 ts_manager.cc:194] Registered new tserver with Master: 1c55820ff11245ecbef4de640c833d1f (127.22.63.65:37737)
I20260806 01:07:24.307061 28822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:40701
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:07:24.344460 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:24.691098 29094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:24.691457 29094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:24.691639 29094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:24.699218 29094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:24.699368 29094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:24.699543 29094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:24.709707 29094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:24.711103 29094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:24.712512 29094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:24.725822 29102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:24.725850 29101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:24.726529 29105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:24.726835 29094 server_base.cc:1034] running on GCE node
I20260806 01:07:24.727499 29094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:24.728510 29094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:24.729717 29094 hybrid_clock.cc:648] HybridClock initialized: now 1785978444729672 us; error 44 us; skew 500 ppm
I20260806 01:07:24.730043 29094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:24 dist-test-slave-7f01 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978444, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:24.733359 29094 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:24.734521 29094 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:24.734658 29094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:24.734983 29094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:24.737952 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "12fdae220ece42218d19a91dcdb3ea9c"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "ee9cf222d5ca73dd381b692cfbd30f48"
server_key_iv: "8753ff7e01c16f5d4403c267cad847c5"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.738773 29094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "12fdae220ece42218d19a91dcdb3ea9c"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "ee9cf222d5ca73dd381b692cfbd30f48"
server_key_iv: "8753ff7e01c16f5d4403c267cad847c5"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.743877 29094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:07:24.747483 29116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.748878 29094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:24.749095 29094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "12fdae220ece42218d19a91dcdb3ea9c"
format_stamp: "Formatted at 2026-08-06 01:07:24 on dist-test-slave-7f01"
server_key: "ee9cf222d5ca73dd381b692cfbd30f48"
server_key_iv: "8753ff7e01c16f5d4403c267cad847c5"
server_key_version: "encryptionkey@0"
I20260806 01:07:24.749331 29094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:24.770116 29094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:24.774662 29094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:24.774993 29094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:24.776084 29094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:24.777515 29094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:24.777644 29094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.777786 29094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:24.777878 29094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:24.812987 29094 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:36139
I20260806 01:07:24.813046 29267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:36139 every 8 connection(s)
I20260806 01:07:24.813884 29094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:24.818670 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 29094
I20260806 01:07:24.818885 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:24.819355 22781 external_mini_cluster.cc:1428] Setting key c4b6d808ffe059f712314306d1f92562
Aug 06 01:07:24 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978444, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:24.836316 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:24.816641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:49799 (local address 127.22.63.126:33117)
0806 01:07:24.817146 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:24.817150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:24.818246 (+ 1096us) server_negotiation.cc:407] Connection header received
0806 01:07:24.820452 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:24.820458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:24.820567 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:24.820687 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:24.822441 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.823181 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.824943 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.825388 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.825530 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:24.829228 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:24.829233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:24.829236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:24.829271 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:24.832383 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:24.833379 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:24.833384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:24.833386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:24.833540 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:24.834047 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:24.834050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:24.834052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:24.834411 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:24.835494 (+ 1083us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:24.835633 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 01:07:24.835907 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":395,"thread_start_us":282,"threads_started":1}
I20260806 01:07:24.837352 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:24.817268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33117 (local address 127.22.63.66:49799)
0806 01:07:24.818037 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:24.818086 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:24.820153 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:24.820941 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:24.820949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:24.821669 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:24.822253 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:24.822270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.823367 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.823372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:24.824780 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:24.824793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:24.825630 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:24.825637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:24.825861 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:24.826578 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:24.826600 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:24.828982 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:24.832539 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:24.832552 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:24.832574 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:24.833230 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:24.833660 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:24.833668 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:24.833674 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:24.833921 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:24.834614 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:24.834625 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:24.835142 (+ 517us) client_negotiation.cc:769] Sending connection context
0806 01:07:24.835492 (+ 350us) client_negotiation.cc:240] Negotiation successful
0806 01:07:24.836131 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":313,"threads_started":1}
I20260806 01:07:24.839021 29270 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33117
I20260806 01:07:24.839383 29270 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:24.840234 29270 heartbeater.cc:511] Master 127.22.63.126:33117 requested a full tablet report, sending...
I20260806 01:07:24.841960 28822 ts_manager.cc:194] Registered new tserver with Master: 12fdae220ece42218d19a91dcdb3ea9c (127.22.63.66:36139)
I20260806 01:07:24.842852 28822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:49799
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:07:24.879583 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:25.226009 29277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:25.226280 29277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:25.226452 29277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:25.233738 29277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:25.233907 29277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:25.234102 29277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:25.244185 29277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:33117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:25.245589 29277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:25.247135 29277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:25.260344 29287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:25.260449 29288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:25.260713 29292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:25.261474 29277 server_base.cc:1034] running on GCE node
I20260806 01:07:25.262133 29277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:25.263191 29277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:25.264467 29277 hybrid_clock.cc:648] HybridClock initialized: now 1785978445264408 us; error 42 us; skew 500 ppm
I20260806 01:07:25.264842 29277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:25 dist-test-slave-7f01 krb5kdc[28762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978445, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:25.268388 29277 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:25.269662 29277 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:25.269810 29277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:25.270159 29277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:25.273341 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "74fbea6c99a04a0a9f2668d484756968"
format_stamp: "Formatted at 2026-08-06 01:07:25 on dist-test-slave-7f01"
server_key: "bbff26d777155fff4853d9340b3109cb"
server_key_iv: "b995ec32c91fb16a731b6a3df5b12ad4"
server_key_version: "encryptionkey@0"
I20260806 01:07:25.274260 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "74fbea6c99a04a0a9f2668d484756968"
format_stamp: "Formatted at 2026-08-06 01:07:25 on dist-test-slave-7f01"
server_key: "bbff26d777155fff4853d9340b3109cb"
server_key_iv: "b995ec32c91fb16a731b6a3df5b12ad4"
server_key_version: "encryptionkey@0"
I20260806 01:07:25.279929 29277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:07:25.283488 29300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:25.284869 29277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:25.285074 29277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "74fbea6c99a04a0a9f2668d484756968"
format_stamp: "Formatted at 2026-08-06 01:07:25 on dist-test-slave-7f01"
server_key: "bbff26d777155fff4853d9340b3109cb"
server_key_iv: "b995ec32c91fb16a731b6a3df5b12ad4"
server_key_version: "encryptionkey@0"
I20260806 01:07:25.285310 29277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:25.300428 29277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:25.304905 29277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:25.305199 29277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:25.306236 29277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:25.307667 29277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:25.307842 29277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:07:25.307976 29277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:25.308068 29277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:25.310808 29087 heartbeater.cc:503] Master 127.22.63.126:33117 was elected leader, sending a full tablet report...
I20260806 01:07:25.345074 29277 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:44597
I20260806 01:07:25.345103 29472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:44597 every 8 connection(s)
I20260806 01:07:25.346011 29277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:25.355469 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 29277
I20260806 01:07:25.355690 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:25.356272 22781 external_mini_cluster.cc:1428] Setting key 91d50cfd5d3f75d56279f31e211b23e1
Aug 06 01:07:25 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978445, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:25.366762 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.348630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:42823 (local address 127.22.63.126:33117)
0806 01:07:25.349122 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.349127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.349931 (+ 804us) server_negotiation.cc:407] Connection header received
0806 01:07:25.351779 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.351784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.351895 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.352020 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:25.353656 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.354521 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.356801 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.357375 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.357545 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.361454 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:25.361459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:25.361462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:25.361502 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:25.364046 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.364709 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.364713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.364715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.364828 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.365260 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.365264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.365266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.365530 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:25.365641 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.366312 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.366498 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":280,"threads_started":1}
I20260806 01:07:25.367725 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.349045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33117 (local address 127.22.63.67:42823)
0806 01:07:25.349758 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.349805 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.351494 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.352257 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.352266 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.352792 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:25.353469 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.353487 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.354672 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.354682 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.356641 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.356659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.357611 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.357616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.357943 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.358786 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:25.358827 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:25.361254 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:25.364171 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:25.364179 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:25.364191 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:25.364610 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:25.364951 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:25.364954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:25.364957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:25.365149 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:25.365640 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:25.365645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:25.365954 (+ 309us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.366190 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.366574 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":297,"threads_started":1}
I20260806 01:07:25.369160 29474 heartbeater.cc:347] Connected to a master server at 127.22.63.126:33117
I20260806 01:07:25.369431 29474 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:25.370097 29474 heartbeater.cc:511] Master 127.22.63.126:33117 requested a full tablet report, sending...
I20260806 01:07:25.371840 28822 ts_manager.cc:194] Registered new tserver with Master: 74fbea6c99a04a0a9f2668d484756968 (127.22.63.67:44597)
I20260806 01:07:25.372692 28822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:42823
I20260806 01:07:25.384668 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:07:25.402482 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.389589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.22.63.126:33117)
0806 01:07:25.389784 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.389791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.390426 (+ 635us) server_negotiation.cc:407] Connection header received
0806 01:07:25.390582 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.390586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.390698 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.390786 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:25.392445 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.393206 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.394481 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.394887 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.394988 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.397364 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:25.397369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:25.397373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:25.397406 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:25.400169 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.400794 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.400798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.400800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.400915 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.401260 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.401263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.401265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.401585 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:25.401764 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.402063 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.402238 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:07:25.406522 28822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:25.413504 28822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:25.443477 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.426658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36139 (local address 127.0.0.1:52356)
0806 01:07:25.427552 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.427612 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.427819 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.428691 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.428699 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.428729 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:25.429167 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.429187 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.434845 (+ 5658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.434854 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.439454 (+ 4600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.439476 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.442194 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.442204 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.442545 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.442637 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.442801 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.443071 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":471,"threads_started":1}
I20260806 01:07:25.444875 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.426738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52356 (local address 127.22.63.66:36139)
0806 01:07:25.428016 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.428027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.428059 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:07:25.428169 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.428179 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.428441 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.430691 (+ 2250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:25.431045 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.434621 (+ 3576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.439768 (+ 5147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.441851 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.444028 (+ 2177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.444261 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.444353 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.444538 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":907,"threads_started":1}
I20260806 01:07:25.448382 29177 tablet_service.cc:1467] Processing CreateTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc (DEFAULT_TABLE table=test-table [id=f49efa7f9134456392e8e128761c56ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:25.449342 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.427206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:44597 (local address 127.0.0.1:52068)
0806 01:07:25.428380 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.428398 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.428518 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.441509 (+ 12991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.441517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.441559 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:25.442159 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.442178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.444861 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.444873 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.446506 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.446523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.448344 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.448355 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.448618 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.448643 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.448757 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.448903 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"mutex_wait_us":52,"thread_start_us":909,"threads_started":1}
I20260806 01:07:25.449337 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.428089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.22.63.67:44597)
0806 01:07:25.440430 (+ 12341us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.440441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.440488 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:07:25.440617 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.440629 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.440982 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.442408 (+ 1426us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:25.442736 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.444684 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.446704 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.448117 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.448270 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.448506 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.448815 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.448973 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12081,"thread_start_us":13092,"threads_started":1}
I20260806 01:07:25.449982 29177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0d495bae5648fc8a5c6b64151bd1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:25.452950 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.428519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:37737 (local address 127.0.0.1:38418)
0806 01:07:25.442773 (+ 14254us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.442794 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.442917 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.444503 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.444508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.444532 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:25.444886 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.444905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.447862 (+ 2957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.447867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.450149 (+ 2282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.450163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.452067 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.452074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.452246 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.452290 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.452394 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.452528 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14103,"mutex_wait_us":28,"thread_start_us":15778,"threads_started":1}
I20260806 01:07:25.453027 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.428982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.22.63.65:37737)
0806 01:07:25.443513 (+ 14531us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.443522 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.443550 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:07:25.443640 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.443648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.443911 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.444109 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:25.446023 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.447687 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.450343 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.451804 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.451976 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.452204 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.452466 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.452638 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14338,"thread_start_us":14183,"threads_started":1}
I20260806 01:07:25.455788 29376 tablet_service.cc:1467] Processing CreateTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc (DEFAULT_TABLE table=test-table [id=f49efa7f9134456392e8e128761c56ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:25.457365 29376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0d495bae5648fc8a5c6b64151bd1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:25.456938 28990 tablet_service.cc:1467] Processing CreateTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc (DEFAULT_TABLE table=test-table [id=f49efa7f9134456392e8e128761c56ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:25.458259 28990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf0d495bae5648fc8a5c6b64151bd1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:25.465152 29501 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Bootstrap starting.
I20260806 01:07:25.469550 29501 tablet_bootstrap.cc:654] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:25.470633 29501 log.cc:824] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:25.472672 29508 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Bootstrap starting.
I20260806 01:07:25.473433 29507 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Bootstrap starting.
I20260806 01:07:25.473855 29501 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: No bootstrap required, opened a new log
I20260806 01:07:25.474144 29501 ts_tablet_manager.cc:1397] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 01:07:25.476255 29507 tablet_bootstrap.cc:654] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:25.476330 29508 tablet_bootstrap.cc:654] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:25.477546 29508 log.cc:824] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:25.477567 29507 log.cc:824] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:25.480669 29508 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: No bootstrap required, opened a new log
I20260806 01:07:25.480829 29507 tablet_bootstrap.cc:492] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: No bootstrap required, opened a new log
I20260806 01:07:25.481011 29508 ts_tablet_manager.cc:1397] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:07:25.481097 29507 ts_tablet_manager.cc:1397] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:07:25.484663 29501 raft_consensus.cc:348] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.485181 29501 raft_consensus.cc:374] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:25.485287 29501 raft_consensus.cc:729] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12fdae220ece42218d19a91dcdb3ea9c, State: Initialized, Role: FOLLOWER
I20260806 01:07:25.486037 29501 consensus_queue.cc:260] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.487317 29270 heartbeater.cc:503] Master 127.22.63.126:33117 was elected leader, sending a full tablet report...
I20260806 01:07:25.488749 29501 ts_tablet_manager.cc:1428] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 01:07:25.490211 29508 raft_consensus.cc:348] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.490702 29508 raft_consensus.cc:374] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:25.490800 29508 raft_consensus.cc:729] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c55820ff11245ecbef4de640c833d1f, State: Initialized, Role: FOLLOWER
I20260806 01:07:25.491429 29507 raft_consensus.cc:348] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.491539 29508 consensus_queue.cc:260] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.491902 29507 raft_consensus.cc:374] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:25.492018 29507 raft_consensus.cc:729] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fbea6c99a04a0a9f2668d484756968, State: Initialized, Role: FOLLOWER
I20260806 01:07:25.492594 29507 consensus_queue.cc:260] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.494163 29508 ts_tablet_manager.cc:1428] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 01:07:25.495775 29474 heartbeater.cc:503] Master 127.22.63.126:33117 was elected leader, sending a full tablet report...
I20260806 01:07:25.496227 29507 ts_tablet_manager.cc:1428] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260806 01:07:25.529290 29513 raft_consensus.cc:482] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:25.529575 29513 raft_consensus.cc:504] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
W20260806 01:07:25.530215 29088 tablet.cc:2369] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:25.531261 29513 leader_election.cc:288] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fbea6c99a04a0a9f2668d484756968 (127.22.63.67:44597), 1c55820ff11245ecbef4de640c833d1f (127.22.63.65:37737)
I20260806 01:07:25.540236 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.532595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:57993 (local address 127.22.63.65:37737)
0806 01:07:25.532801 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.532808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.533032 (+ 224us) server_negotiation.cc:407] Connection header received
0806 01:07:25.533197 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.533201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.533259 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.533334 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:25.534031 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.535085 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.537730 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.538997 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.539166 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.539243 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.539840 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.539990 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:07:25.540234 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.531606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:44233 (local address 127.22.63.67:44597)
0806 01:07:25.531914 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.531923 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.532410 (+ 487us) server_negotiation.cc:407] Connection header received
0806 01:07:25.532719 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.532726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.532836 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.532908 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:25.533738 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.534878 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.537739 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.539012 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.539169 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.539243 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.539840 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.539972 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:07:25.540477 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.532295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:37737 (local address 127.22.63.66:57993)
0806 01:07:25.532935 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.532951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.533076 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.533379 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.533383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.533410 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:25.533843 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.533852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.535215 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.535221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.537548 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.537564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.539300 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.539304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.539569 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.539694 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.539812 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.539986 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":147,"thread_start_us":299,"threads_started":1}
I20260806 01:07:25.540817 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.531582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:44597 (local address 127.22.63.66:44233)
0806 01:07:25.532279 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:25.532300 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:25.532499 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:25.533128 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:25.533133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:25.533162 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:25.533577 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.533590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.535010 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.535013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.537549 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:25.537564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.539295 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.539302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:25.539569 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.539697 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 01:07:25.539813 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 01:07:25.539986 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":175,"thread_start_us":348,"threads_started":1}
I20260806 01:07:25.541515 29013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0d495bae5648fc8a5c6b64151bd1fc" candidate_uuid: "12fdae220ece42218d19a91dcdb3ea9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c55820ff11245ecbef4de640c833d1f" is_pre_election: true
I20260806 01:07:25.541847 29409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0d495bae5648fc8a5c6b64151bd1fc" candidate_uuid: "12fdae220ece42218d19a91dcdb3ea9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fbea6c99a04a0a9f2668d484756968" is_pre_election: true
I20260806 01:07:25.542021 29013 raft_consensus.cc:2454] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fdae220ece42218d19a91dcdb3ea9c in term 0.
I20260806 01:07:25.542352 29409 raft_consensus.cc:2454] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fdae220ece42218d19a91dcdb3ea9c in term 0.
I20260806 01:07:25.542773 29122 leader_election.cc:302] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fdae220ece42218d19a91dcdb3ea9c, 1c55820ff11245ecbef4de640c833d1f; no voters:
I20260806 01:07:25.543192 29513 raft_consensus.cc:2781] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:25.543316 29513 raft_consensus.cc:482] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:25.543460 29513 raft_consensus.cc:3037] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:25.545751 29513 raft_consensus.cc:504] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.546360 29513 leader_election.cc:288] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [CANDIDATE]: Term 1 election: Requested vote from peers 74fbea6c99a04a0a9f2668d484756968 (127.22.63.67:44597), 1c55820ff11245ecbef4de640c833d1f (127.22.63.65:37737)
I20260806 01:07:25.546979 29409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0d495bae5648fc8a5c6b64151bd1fc" candidate_uuid: "12fdae220ece42218d19a91dcdb3ea9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fbea6c99a04a0a9f2668d484756968"
I20260806 01:07:25.546976 29013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf0d495bae5648fc8a5c6b64151bd1fc" candidate_uuid: "12fdae220ece42218d19a91dcdb3ea9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c55820ff11245ecbef4de640c833d1f"
I20260806 01:07:25.547195 29409 raft_consensus.cc:3037] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:25.547195 29013 raft_consensus.cc:3037] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:25.550506 29409 raft_consensus.cc:2454] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fdae220ece42218d19a91dcdb3ea9c in term 1.
I20260806 01:07:25.550506 29013 raft_consensus.cc:2454] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fdae220ece42218d19a91dcdb3ea9c in term 1.
I20260806 01:07:25.550935 29122 leader_election.cc:302] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fdae220ece42218d19a91dcdb3ea9c, 74fbea6c99a04a0a9f2668d484756968; no voters:
I20260806 01:07:25.551218 29513 raft_consensus.cc:2781] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:25.551707 29513 raft_consensus.cc:686] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 1 LEADER]: Becoming Leader. State: Replica: 12fdae220ece42218d19a91dcdb3ea9c, State: Running, Role: LEADER
I20260806 01:07:25.552313 29513 consensus_queue.cc:237] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } }
I20260806 01:07:25.557526 28821 catalog_manager.cc:5495] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 12fdae220ece42218d19a91dcdb3ea9c (127.22.63.66). New cstate: current_term: 1 leader_uuid: "12fdae220ece42218d19a91dcdb3ea9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fdae220ece42218d19a91dcdb3ea9c" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 } health_report { overall_health: UNKNOWN } } }
W20260806 01:07:25.565843 29271 tablet.cc:2369] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:25.592855 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.587447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.22.63.66:36139)
0806 01:07:25.587633 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.587637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.587655 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:07:25.587759 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.587763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.587850 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.587986 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:25.588345 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.589203 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.590695 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.591225 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.591322 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.591818 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:25.592360 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:25.592479 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.592572 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.592639 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 01:07:25.597574 29475 tablet.cc:2369] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:25.601476 29409 raft_consensus.cc:1261] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 1 FOLLOWER]: Refusing update from remote peer 12fdae220ece42218d19a91dcdb3ea9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:25.601475 29013 raft_consensus.cc:1261] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 1 FOLLOWER]: Refusing update from remote peer 12fdae220ece42218d19a91dcdb3ea9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:25.602435 29513 consensus_queue.cc:1035] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c55820ff11245ecbef4de640c833d1f" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:25.602782 29520 consensus_queue.cc:1035] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fbea6c99a04a0a9f2668d484756968" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:25.614681 29523 mvcc.cc:205] Tried to move back new op lower bound from 7315367713173745664 to 7315367712986869760. Current Snapshot: MvccSnapshot[applied={T|T < 7315367713173745664}]
I20260806 01:07:25.630823 28821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:25.631131 28821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:25.634953 28821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65654f01aaff4f8da8b50c6c6f98ba6d: Sending DeleteTablet for 3 replicas of tablet cf0d495bae5648fc8a5c6b64151bd1fc
I20260806 01:07:25.636153 29177 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:25 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:52356
I20260806 01:07:25.636363 29376 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:25 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:52068
I20260806 01:07:25.636463 28990 tablet_service.cc:1514] Processing DeleteTablet for tablet cf0d495bae5648fc8a5c6b64151bd1fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:25 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:38418
I20260806 01:07:25.638932 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28907
I20260806 01:07:25.647995 29538 tablet_replica.cc:330] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: stopping tablet replica
I20260806 01:07:25.648675 29538 raft_consensus.cc:2229] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:25.649384 29538 raft_consensus.cc:2258] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:25.650094 29539 tablet_replica.cc:330] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: stopping tablet replica
I20260806 01:07:25.651191 29539 raft_consensus.cc:2229] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:25.652943 29539 raft_consensus.cc:2258] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:25.654384 29538 ts_tablet_manager.cc:1905] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:25.657083 29539 ts_tablet_manager.cc:1905] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:25.657743 29537 tablet_replica.cc:330] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: stopping tablet replica
Aug 06 01:07:25 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:25.659464 29537 raft_consensus.cc:2229] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:25.665338 29539 ts_tablet_manager.cc:1918] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:25.665608 29539 log.cc:1196] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/cf0d495bae5648fc8a5c6b64151bd1fc
I20260806 01:07:25.665807 29538 ts_tablet_manager.cc:1918] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:25.666044 29538 log.cc:1196] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/cf0d495bae5648fc8a5c6b64151bd1fc
I20260806 01:07:25.666507 29539 ts_tablet_manager.cc:1939] T cf0d495bae5648fc8a5c6b64151bd1fc P 1c55820ff11245ecbef4de640c833d1f: Deleting consensus metadata
I20260806 01:07:25.666637 29538 ts_tablet_manager.cc:1939] T cf0d495bae5648fc8a5c6b64151bd1fc P 74fbea6c99a04a0a9f2668d484756968: Deleting consensus metadata
I20260806 01:07:25.668439 29537 raft_consensus.cc:2258] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:25.668733 28805 catalog_manager.cc:4841] TS 1c55820ff11245ecbef4de640c833d1f (127.22.63.65:37737): tablet cf0d495bae5648fc8a5c6b64151bd1fc (table test-table [id=f49efa7f9134456392e8e128761c56ec]) successfully deleted
I20260806 01:07:25.670164 28805 catalog_manager.cc:4841] TS 74fbea6c99a04a0a9f2668d484756968 (127.22.63.67:44597): tablet cf0d495bae5648fc8a5c6b64151bd1fc (table test-table [id=f49efa7f9134456392e8e128761c56ec]) successfully deleted
I20260806 01:07:25.670918 29537 ts_tablet_manager.cc:1905] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:25.676532 29500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.640948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.22.63.65:37737)
0806 01:07:25.641198 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.641207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.645036 (+ 3829us) server_negotiation.cc:407] Connection header received
0806 01:07:25.645257 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.645266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.645400 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.645533 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:25.651464 (+ 5931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.652578 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.654607 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.655244 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.655348 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.668004 (+ 12656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:25.668022 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:25.668039 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:25.668089 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:25.672332 (+ 4243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.673269 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.673279 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.673283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.673454 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.673899 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.673904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.673908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.674351 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:25.674670 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.675924 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.676151 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:07:25.677240 29082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:25.678565 29537 ts_tablet_manager.cc:1918] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:25.678747 29537 log.cc:1196] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/cf0d495bae5648fc8a5c6b64151bd1fc
I20260806 01:07:25.679359 29537 ts_tablet_manager.cc:1939] T cf0d495bae5648fc8a5c6b64151bd1fc P 12fdae220ece42218d19a91dcdb3ea9c: Deleting consensus metadata
I20260806 01:07:25.681300 28804 catalog_manager.cc:4841] TS 12fdae220ece42218d19a91dcdb3ea9c (127.22.63.66:36139): tablet cf0d495bae5648fc8a5c6b64151bd1fc (table test-table [id=f49efa7f9134456392e8e128761c56ec]) successfully deleted
I20260806 01:07:25.785915 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28907
I20260806 01:07:25.813261 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 29094
Aug 06 01:07:25 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:07:25.833858 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.814079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.22.63.66:36139)
0806 01:07:25.814325 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.814331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.814441 (+ 110us) server_negotiation.cc:407] Connection header received
0806 01:07:25.814631 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.814636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.814740 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.814859 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:25.816636 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.817987 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.819944 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.820651 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.820790 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.826113 (+ 5323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:25.826128 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:25.826146 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:25.826193 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:25.830579 (+ 4386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.831390 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.831399 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.831404 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.831570 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.832045 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.832052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.832061 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.832513 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:25.832825 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.833149 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.833439 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 01:07:25.834944 29260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:25.952898 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 29094
I20260806 01:07:25.982404 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 29277
Aug 06 01:07:25 dist-test-slave-7f01 krb5kdc[28762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:07:25.998811 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:25.983298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.22.63.67:44597)
0806 01:07:25.983496 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:25.983504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:25.983571 (+ 67us) server_negotiation.cc:407] Connection header received
0806 01:07:25.983799 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:25.983807 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:25.983899 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:25.984018 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:25.985335 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.986118 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.987344 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:25.987874 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:25.988020 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:25.992802 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:25.992817 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:25.992835 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:25.992878 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:25.995721 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.996514 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.996523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.996525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.996739 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:25.997244 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:25.997248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:25.997250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:25.997655 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:25.997960 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:25.998243 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 01:07:25.998427 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:07:25.999480 29466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:26.091144 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 29277
I20260806 01:07:26.115828 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 28778
I20260806 01:07:26.117120 28863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:26.228762 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 28778
2026-08-06T01:07:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4545 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T01:07:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:26Z Disabled control of system clock
I20260806 01:07:26.312279 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45243
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:07:26.652091 29549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:26.652396 29549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:26.652501 29549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:26.661978 29549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:26.662177 29549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:26.662250 29549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:26.662343 29549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:26.662418 29549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:26.675805 29549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:26.677281 29549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:26.679272 29549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:26.691944 29556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:26.691998 29558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:26.691954 29562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:26.693305 29549 server_base.cc:1034] running on GCE node
I20260806 01:07:26.694175 29549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:26.695780 29549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:26.697041 29549 hybrid_clock.cc:648] HybridClock initialized: now 1785978446696999 us; error 38 us; skew 500 ppm
I20260806 01:07:26.697412 29549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:26.702831 29549 webserver.cc:460] Webserver started at http://127.22.63.126:37521/ using document root <none> and password file <none>
I20260806 01:07:26.703600 29549 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:26.703807 29549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:26.704181 29549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:26.707221 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "a6dfef68792044f68d45ec0e21de2d2c"
format_stamp: "Formatted at 2026-08-06 01:07:26 on dist-test-slave-7f01"
server_key: "1a1d8a8023c4373b00c764cd94a243f9"
server_key_iv: "df5529541a5c1ac4d5657fbd22c25b25"
server_key_version: "encryptionkey@0"
I20260806 01:07:26.708112 29549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "a6dfef68792044f68d45ec0e21de2d2c"
format_stamp: "Formatted at 2026-08-06 01:07:26 on dist-test-slave-7f01"
server_key: "1a1d8a8023c4373b00c764cd94a243f9"
server_key_iv: "df5529541a5c1ac4d5657fbd22c25b25"
server_key_version: "encryptionkey@0"
I20260806 01:07:26.713366 29549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:07:26.716945 29569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:26.718394 29549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:07:26.718592 29549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a6dfef68792044f68d45ec0e21de2d2c"
format_stamp: "Formatted at 2026-08-06 01:07:26 on dist-test-slave-7f01"
server_key: "1a1d8a8023c4373b00c764cd94a243f9"
server_key_iv: "df5529541a5c1ac4d5657fbd22c25b25"
server_key_version: "encryptionkey@0"
I20260806 01:07:26.718827 29549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:26.725028 29549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:26.725818 29549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:26.726137 29549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:26.745941 29549 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:45243
I20260806 01:07:26.745981 29642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:45243 every 8 connection(s)
I20260806 01:07:26.747362 29549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:26.751242 29643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:26.755046 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 29549
I20260806 01:07:26.755259 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:26.755980 22781 external_mini_cluster.cc:1428] Setting key 3037a0aa09ee1d112aed4ee7be8869d3
I20260806 01:07:26.762732 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: Bootstrap starting.
I20260806 01:07:26.764695 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:26.758075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.22.63.126:45243)
0806 01:07:26.759128 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:26.759138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:26.759179 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:07:26.761100 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:26.761124 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:26.761521 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:26.761838 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:26.763038 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:26.763045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:26.763058 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:26.763072 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:26.763133 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:26.763148 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:26.763328 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:26.763474 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 01:07:26.763733 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":411,"threads_started":1}
I20260806 01:07:26.766206 29643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:26.767414 29643 log.cc:824] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:26.770183 29643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: No bootstrap required, opened a new log
I20260806 01:07:26.778141 29643 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } }
I20260806 01:07:26.778483 29643 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:26.778555 29643 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6dfef68792044f68d45ec0e21de2d2c, State: Initialized, Role: FOLLOWER
I20260806 01:07:26.779049 29643 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } }
I20260806 01:07:26.779311 29643 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:26.779459 29643 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:26.779691 29643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:26.782080 29643 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } }
I20260806 01:07:26.782630 29643 leader_election.cc:302] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6dfef68792044f68d45ec0e21de2d2c; no voters:
I20260806 01:07:26.783245 29643 leader_election.cc:288] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:26.783382 29650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:26.785373 29643 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:26.785825 29650 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [term 1 LEADER]: Becoming Leader. State: Replica: a6dfef68792044f68d45ec0e21de2d2c, State: Running, Role: LEADER
I20260806 01:07:26.786361 29650 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } }
I20260806 01:07:26.789386 29651 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6dfef68792044f68d45ec0e21de2d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } } }
I20260806 01:07:26.789757 29651 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:26.790227 29654 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6dfef68792044f68d45ec0e21de2d2c. Latest consensus state: current_term: 1 leader_uuid: "a6dfef68792044f68d45ec0e21de2d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6dfef68792044f68d45ec0e21de2d2c" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45243 } } }
I20260806 01:07:26.790576 29654 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:26.791400 29665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:26.796108 29665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:26.805047 29665 catalog_manager.cc:1347] Generated new cluster ID: c7706ee90c774c3db29d9ab6f8cab5f1
I20260806 01:07:26.805181 29665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:26.814770 29665 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:26.815253 29665 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:26.824954 29665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: Generated new TSK 0
I20260806 01:07:26.825968 29665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:07:26.838800 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--builtin_ntp_servers=127.22.63.84:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 01:07:27.191279 29673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:27.191568 29673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:27.191722 29673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:27.199156 29673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:27.199262 29673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:27.199446 29673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:27.208757 29673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:27.210162 29673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:27.212169 29673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:27.225979 29678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:27.225950 29682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:27.225905 29679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:27.227165 29673 server_base.cc:1034] running on GCE node
I20260806 01:07:27.227890 29673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:27.228966 29673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:27.230206 29673 hybrid_clock.cc:648] HybridClock initialized: now 1785978447230125 us; error 87 us; skew 500 ppm
I20260806 01:07:27.230540 29673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:27.233003 29673 webserver.cc:460] Webserver started at http://127.22.63.65:33325/ using document root <none> and password file <none>
I20260806 01:07:27.233721 29673 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:27.233860 29673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:27.234222 29673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:27.237218 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "efaf5e68ae1a470997f14d5b31ff7315"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "1fbdb682154931d47903d424f11d76b3"
server_key_iv: "33e2021895e688078ab6fcf3a2cd1722"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.238054 29673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "efaf5e68ae1a470997f14d5b31ff7315"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "1fbdb682154931d47903d424f11d76b3"
server_key_iv: "33e2021895e688078ab6fcf3a2cd1722"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.243402 29673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:27.247082 29697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.248502 29673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:07:27.248689 29673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "efaf5e68ae1a470997f14d5b31ff7315"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "1fbdb682154931d47903d424f11d76b3"
server_key_iv: "33e2021895e688078ab6fcf3a2cd1722"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.248941 29673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:27.255012 29673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:27.255967 29673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:27.256254 29673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:27.257223 29673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:27.258673 29673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:27.258810 29673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.258946 29673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:27.259037 29673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.294822 29673 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:40377
I20260806 01:07:27.294921 29854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:40377 every 8 connection(s)
I20260806 01:07:27.296240 29673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:27.304625 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 29673
I20260806 01:07:27.304804 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:27.298986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:49313 (local address 127.22.63.126:45243)
0806 01:07:27.299455 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:27.299460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:27.300606 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 01:07:27.302358 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:27.302370 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:27.302449 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:27.302573 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:27.304000 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:27.304006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:27.304011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:27.304037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:27.304080 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:27.304091 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:27.304231 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:27.304431 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 01:07:27.304503 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":262,"threads_started":1}
I20260806 01:07:27.304888 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:27.305611 22781 external_mini_cluster.cc:1428] Setting key 35979ca83f631bfe5329fe0edb375c99
I20260806 01:07:27.305800 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:27.299429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45243 (local address 127.22.63.65:49313)
0806 01:07:27.300440 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:27.300478 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:27.302082 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:27.302839 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:27.302847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:27.302940 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:27.303637 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:27.303639 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:27.303678 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:27.303827 (+ 149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:27.304190 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:27.304204 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:27.304213 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 01:07:27.304526 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 01:07:27.304781 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":356,"threads_started":1}
I20260806 01:07:27.307478 29855 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45243
I20260806 01:07:27.307883 29855 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:27.308607 29855 heartbeater.cc:511] Master 127.22.63.126:45243 requested a full tablet report, sending...
I20260806 01:07:27.309193 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--builtin_ntp_servers=127.22.63.84:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 01:07:27.310900 29593 ts_manager.cc:194] Registered new tserver with Master: efaf5e68ae1a470997f14d5b31ff7315 (127.22.63.65:40377)
W20260806 01:07:27.646920 29861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:27.647203 29861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:27.647292 29861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:27.654685 29861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:27.654789 29861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:27.654904 29861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:27.664000 29861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:27.665277 29861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:27.667097 29861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:27.681085 29868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:27.681114 29870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:27.681203 29873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:27.681905 29861 server_base.cc:1034] running on GCE node
I20260806 01:07:27.682613 29861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:27.683597 29861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:27.684800 29861 hybrid_clock.cc:648] HybridClock initialized: now 1785978447684760 us; error 90 us; skew 500 ppm
I20260806 01:07:27.685148 29861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:27.687566 29861 webserver.cc:460] Webserver started at http://127.22.63.66:32911/ using document root <none> and password file <none>
I20260806 01:07:27.688331 29861 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:27.688424 29861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:27.688810 29861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:27.691654 29861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "a6809b102b684170b4282dc608d6cda0"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "e295f5d52573bfec25a2fce0ace4d144"
server_key_iv: "ca13187b51ae6c1a051fa30a868e3a0d"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.692539 29861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "a6809b102b684170b4282dc608d6cda0"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "e295f5d52573bfec25a2fce0ace4d144"
server_key_iv: "ca13187b51ae6c1a051fa30a868e3a0d"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.697682 29861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:27.701220 29880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.702534 29861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:27.702710 29861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "a6809b102b684170b4282dc608d6cda0"
format_stamp: "Formatted at 2026-08-06 01:07:27 on dist-test-slave-7f01"
server_key: "e295f5d52573bfec25a2fce0ace4d144"
server_key_iv: "ca13187b51ae6c1a051fa30a868e3a0d"
server_key_version: "encryptionkey@0"
I20260806 01:07:27.702942 29861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:27.708798 29861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:27.709599 29861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:27.709887 29861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:27.710806 29861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:27.712281 29861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:27.712409 29861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.712550 29861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:27.712643 29861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:27.747622 29861 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:43481
I20260806 01:07:27.747663 30040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:43481 every 8 connection(s)
I20260806 01:07:27.749046 29861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:27.754410 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 29861
I20260806 01:07:27.754571 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:27.755118 22781 external_mini_cluster.cc:1428] Setting key c8bfdfff0f5995c60f88d6ca86cefb6e
I20260806 01:07:27.758140 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--builtin_ntp_servers=127.22.63.84:46471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 01:07:27.761108 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:27.751934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:44525 (local address 127.22.63.126:45243)
0806 01:07:27.752123 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:27.752128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:27.753665 (+ 1537us) server_negotiation.cc:407] Connection header received
0806 01:07:27.755611 (+ 1946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:27.755618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:27.755716 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:27.755868 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:27.760328 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:27.760333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:27.760336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:27.760354 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:27.760388 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:27.760400 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:27.760530 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:27.760737 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 01:07:27.760809 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:07:27.762156 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:27.752448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45243 (local address 127.22.63.66:44525)
0806 01:07:27.753512 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:27.753548 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:27.755343 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:27.756193 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:27.756205 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:27.756314 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:27.759994 (+ 3680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:27.759996 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:27.760035 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:27.760125 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:27.760504 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:27.760515 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:27.760523 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:07:27.760837 (+ 314us) client_negotiation.cc:240] Negotiation successful
0806 01:07:27.761110 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":380,"threads_started":1}
I20260806 01:07:27.763571 30041 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45243
I20260806 01:07:27.763929 30041 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:27.764515 30041 heartbeater.cc:511] Master 127.22.63.126:45243 requested a full tablet report, sending...
I20260806 01:07:27.765964 29593 ts_manager.cc:194] Registered new tserver with Master: a6809b102b684170b4282dc608d6cda0 (127.22.63.66:43481)
W20260806 01:07:28.102056 30044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:28.102345 30044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:28.102530 30044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:28.109758 30044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:28.109849 30044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:28.110054 30044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:28.118912 30044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:28.120329 30044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:28.122153 30044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:28.136353 30050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:28.136399 30051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:28.136363 30054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:28.137583 30044 server_base.cc:1034] running on GCE node
I20260806 01:07:28.138134 30044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:28.139055 30044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:28.140259 30044 hybrid_clock.cc:648] HybridClock initialized: now 1785978448140220 us; error 33 us; skew 500 ppm
I20260806 01:07:28.140599 30044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:28.142833 30044 webserver.cc:460] Webserver started at http://127.22.63.67:36213/ using document root <none> and password file <none>
I20260806 01:07:28.143549 30044 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:28.143666 30044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:28.144114 30044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:28.147059 30044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "911931f0e74444ab9fb2a9faaf1dbd7f"
format_stamp: "Formatted at 2026-08-06 01:07:28 on dist-test-slave-7f01"
server_key: "1fb34bf6a1bf4f817f6c129606329701"
server_key_iv: "22896fe8096c77311178bddaf1fe7f12"
server_key_version: "encryptionkey@0"
I20260806 01:07:28.147930 30044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "911931f0e74444ab9fb2a9faaf1dbd7f"
format_stamp: "Formatted at 2026-08-06 01:07:28 on dist-test-slave-7f01"
server_key: "1fb34bf6a1bf4f817f6c129606329701"
server_key_iv: "22896fe8096c77311178bddaf1fe7f12"
server_key_version: "encryptionkey@0"
I20260806 01:07:28.153033 30044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260806 01:07:28.156544 30060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:28.157823 30044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:28.157985 30044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "911931f0e74444ab9fb2a9faaf1dbd7f"
format_stamp: "Formatted at 2026-08-06 01:07:28 on dist-test-slave-7f01"
server_key: "1fb34bf6a1bf4f817f6c129606329701"
server_key_iv: "22896fe8096c77311178bddaf1fe7f12"
server_key_version: "encryptionkey@0"
I20260806 01:07:28.158214 30044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:28.163959 30044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:28.164739 30044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:28.165021 30044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:28.165932 30044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:28.167403 30044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:28.167537 30044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:28.167666 30044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:28.167845 30044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:28.201288 30044 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:46589
I20260806 01:07:28.201340 30225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:46589 every 8 connection(s)
I20260806 01:07:28.202643 30044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:28.204460 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30044
I20260806 01:07:28.204651 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:28.205176 22781 external_mini_cluster.cc:1428] Setting key 359961dc8b9565ab554638bc2c18bd2b
I20260806 01:07:28.211089 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.205275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:51465 (local address 127.22.63.126:45243)
0806 01:07:28.205497 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.205504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.207218 (+ 1714us) server_negotiation.cc:407] Connection header received
0806 01:07:28.209117 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.209121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.209180 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.209260 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.210510 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.210513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.210516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.210530 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.210557 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.210566 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.210651 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.210820 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.210868 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:07:28.212087 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.205874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45243 (local address 127.22.63.67:51465)
0806 01:07:28.207056 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.207102 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.208876 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.209462 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.209469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.209554 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.210251 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.210253 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.210290 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.210372 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.210648 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.210655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.210661 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.210866 (+ 205us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.211078 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":402,"threads_started":1}
I20260806 01:07:28.213356 30226 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45243
I20260806 01:07:28.213644 30226 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:28.214232 30226 heartbeater.cc:511] Master 127.22.63.126:45243 requested a full tablet report, sending...
I20260806 01:07:28.215590 29593 ts_manager.cc:194] Registered new tserver with Master: 911931f0e74444ab9fb2a9faaf1dbd7f (127.22.63.67:46589)
I20260806 01:07:28.220556 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:07:28.227999 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.225286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.22.63.126:45243)
0806 01:07:28.225453 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.225474 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.225938 (+ 464us) server_negotiation.cc:407] Connection header received
0806 01:07:28.226162 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.226167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.226216 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.226314 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.227292 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.227298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.227302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.227320 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.227359 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.227370 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.227503 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.227632 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.227695 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:07:28.232127 29593 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:28.238842 29593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:28.266252 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.251857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.22.63.65:40377)
0806 01:07:28.262766 (+ 10909us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.262774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.262798 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:07:28.262862 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.262868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.263124 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.263866 (+ 742us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.265281 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.265288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.265297 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.265323 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.265375 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.265395 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.265542 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.265753 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.265870 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10748,"thread_start_us":10613,"threads_started":1}
I20260806 01:07:28.266969 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.254513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45836 (local address 127.22.63.66:43481)
0806 01:07:28.261762 (+ 7249us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.261776 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.262722 (+ 946us) server_negotiation.cc:407] Connection header received
0806 01:07:28.262801 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.262808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.263008 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.263861 (+ 853us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.265274 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.265282 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.265297 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.265323 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.265375 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.265408 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.265542 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.266568 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.266686 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7016,"thread_start_us":6866,"threads_started":1}
I20260806 01:07:28.266991 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.251502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.22.63.67:46589)
0806 01:07:28.262241 (+ 10739us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.262253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.262302 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:07:28.262392 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.262404 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.262681 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.264519 (+ 1838us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.265363 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.265373 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.265387 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.265404 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.265481 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.265503 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.265730 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.266532 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.266624 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10518,"thread_start_us":10299,"threads_started":1}
I20260806 01:07:28.267217 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.251120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46589 (local address 127.0.0.1:38762)
0806 01:07:28.253055 (+ 1935us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.253102 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.253304 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.262903 (+ 9599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.262913 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.263035 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.264315 (+ 1280us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.264317 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.264331 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.264369 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.265693 (+ 1324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.265698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.265704 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.265879 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.265987 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1662,"thread_start_us":2427,"threads_started":1}
I20260806 01:07:28.267254 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.251224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58072)
0806 01:07:28.253886 (+ 2662us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.253908 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.254026 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.263301 (+ 9275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.263306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.263330 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.264239 (+ 909us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.264245 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.264291 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.264373 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.265517 (+ 1144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.265531 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.265542 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.265737 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.265830 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2508,"thread_start_us":1401,"threads_started":1}
I20260806 01:07:28.267493 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.254322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:43481 (local address 127.0.0.1:45836)
0806 01:07:28.262462 (+ 8140us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.262481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.262598 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.263301 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.263306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.263330 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.264280 (+ 950us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.264282 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.264294 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.264373 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.265796 (+ 1423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.265801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.265805 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.265961 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.266015 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8001,"thread_start_us":7859,"threads_started":1}
I20260806 01:07:28.270843 29767 tablet_service.cc:1467] Processing CreateTablet for tablet 820f685ab435493c8535ad9067bf595c (DEFAULT_TABLE table=test-table [id=0d9a42ab7e0c47b4802ebfbb25944d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:28.271122 30127 tablet_service.cc:1467] Processing CreateTablet for tablet 820f685ab435493c8535ad9067bf595c (DEFAULT_TABLE table=test-table [id=0d9a42ab7e0c47b4802ebfbb25944d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:28.271412 29944 tablet_service.cc:1467] Processing CreateTablet for tablet 820f685ab435493c8535ad9067bf595c (DEFAULT_TABLE table=test-table [id=0d9a42ab7e0c47b4802ebfbb25944d1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:28.272255 29767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f685ab435493c8535ad9067bf595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:28.272562 30127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f685ab435493c8535ad9067bf595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:28.272660 29944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 820f685ab435493c8535ad9067bf595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:28.284304 30253 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Bootstrap starting.
I20260806 01:07:28.287489 30256 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Bootstrap starting.
I20260806 01:07:28.288287 30254 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Bootstrap starting.
I20260806 01:07:28.288350 30253 tablet_bootstrap.cc:654] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:28.289788 30253 log.cc:824] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:28.292573 30256 tablet_bootstrap.cc:654] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:28.293138 30254 tablet_bootstrap.cc:654] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:28.293851 30256 log.cc:824] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:28.294180 30254 log.cc:824] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:28.294866 30253 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: No bootstrap required, opened a new log
I20260806 01:07:28.295231 30253 ts_tablet_manager.cc:1397] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260806 01:07:28.303279 30256 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: No bootstrap required, opened a new log
I20260806 01:07:28.303645 30256 ts_tablet_manager.cc:1397] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260806 01:07:28.312959 30253 raft_consensus.cc:348] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.313355 30253 raft_consensus.cc:374] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:28.313452 30253 raft_consensus.cc:729] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efaf5e68ae1a470997f14d5b31ff7315, State: Initialized, Role: FOLLOWER
I20260806 01:07:28.313844 29855 heartbeater.cc:503] Master 127.22.63.126:45243 was elected leader, sending a full tablet report...
I20260806 01:07:28.314268 30253 consensus_queue.cc:260] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.314590 30254 tablet_bootstrap.cc:492] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: No bootstrap required, opened a new log
I20260806 01:07:28.314864 30254 ts_tablet_manager.cc:1397] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.016s
I20260806 01:07:28.316148 30256 raft_consensus.cc:348] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.316586 30256 raft_consensus.cc:374] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:28.316676 30256 raft_consensus.cc:729] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911931f0e74444ab9fb2a9faaf1dbd7f, State: Initialized, Role: FOLLOWER
I20260806 01:07:28.317364 30256 consensus_queue.cc:260] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.323190 30253 ts_tablet_manager.cc:1428] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Time spent starting tablet: real 0.028s user 0.018s sys 0.004s
I20260806 01:07:28.324501 30226 heartbeater.cc:503] Master 127.22.63.126:45243 was elected leader, sending a full tablet report...
I20260806 01:07:28.325162 30256 ts_tablet_manager.cc:1428] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Time spent starting tablet: real 0.021s user 0.009s sys 0.011s
I20260806 01:07:28.327415 30254 raft_consensus.cc:348] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.327845 30254 raft_consensus.cc:374] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:28.327960 30254 raft_consensus.cc:729] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6809b102b684170b4282dc608d6cda0, State: Initialized, Role: FOLLOWER
I20260806 01:07:28.328708 30254 consensus_queue.cc:260] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.329908 30041 heartbeater.cc:503] Master 127.22.63.126:45243 was elected leader, sending a full tablet report...
I20260806 01:07:28.330152 30260 raft_consensus.cc:482] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:28.330435 30254 ts_tablet_manager.cc:1428] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Time spent starting tablet: real 0.015s user 0.015s sys 0.002s
I20260806 01:07:28.330538 30260 raft_consensus.cc:504] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.332680 30260 leader_election.cc:288] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 911931f0e74444ab9fb2a9faaf1dbd7f (127.22.63.67:46589), a6809b102b684170b4282dc608d6cda0 (127.22.63.66:43481)
I20260806 01:07:28.336258 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.333628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:50337 (local address 127.22.63.67:46589)
0806 01:07:28.333856 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.333862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.333925 (+ 63us) server_negotiation.cc:407] Connection header received
0806 01:07:28.334132 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.334139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.334201 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.334301 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.335577 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.335586 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.335594 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.335613 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.335656 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.335669 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.335823 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.335917 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.335982 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 01:07:28.336373 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.333043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46589 (local address 127.22.63.65:50337)
0806 01:07:28.333839 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.333860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.333986 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.334477 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.334482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.334509 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.335359 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.335362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.335380 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.335422 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.335794 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.335801 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.335808 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.335979 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.336025 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":397,"threads_started":1}
I20260806 01:07:28.336915 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.334300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:45935 (local address 127.22.63.66:43481)
0806 01:07:28.334470 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.334475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.334647 (+ 172us) server_negotiation.cc:407] Connection header received
0806 01:07:28.334855 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.334860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.334914 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.335030 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.336209 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.336214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.336218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.336238 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.336278 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.336294 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.336384 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.336598 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.336659 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:07:28.337081 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.333957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:43481 (local address 127.22.63.65:45935)
0806 01:07:28.334536 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.334559 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:28.334733 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:28.335189 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:28.335195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:28.335227 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:28.335984 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:07:28.335987 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:28.336003 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:07:28.336045 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:28.336482 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:28.336487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:28.336492 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:07:28.336612 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 01:07:28.336650 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":514,"threads_started":1}
I20260806 01:07:28.337503 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f685ab435493c8535ad9067bf595c" candidate_uuid: "efaf5e68ae1a470997f14d5b31ff7315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" is_pre_election: true
I20260806 01:07:28.338042 30159 raft_consensus.cc:2454] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efaf5e68ae1a470997f14d5b31ff7315 in term 0.
I20260806 01:07:28.338035 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f685ab435493c8535ad9067bf595c" candidate_uuid: "efaf5e68ae1a470997f14d5b31ff7315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6809b102b684170b4282dc608d6cda0" is_pre_election: true
I20260806 01:07:28.338563 29975 raft_consensus.cc:2454] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efaf5e68ae1a470997f14d5b31ff7315 in term 0.
I20260806 01:07:28.338670 29699 leader_election.cc:302] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 911931f0e74444ab9fb2a9faaf1dbd7f, efaf5e68ae1a470997f14d5b31ff7315; no voters:
I20260806 01:07:28.338994 30260 raft_consensus.cc:2781] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:28.339143 30260 raft_consensus.cc:482] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:28.339257 30260 raft_consensus.cc:3037] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:28.341554 30260 raft_consensus.cc:504] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.342437 30260 leader_election.cc:288] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [CANDIDATE]: Term 1 election: Requested vote from peers 911931f0e74444ab9fb2a9faaf1dbd7f (127.22.63.67:46589), a6809b102b684170b4282dc608d6cda0 (127.22.63.66:43481)
I20260806 01:07:28.343024 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f685ab435493c8535ad9067bf595c" candidate_uuid: "efaf5e68ae1a470997f14d5b31ff7315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f"
I20260806 01:07:28.343041 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820f685ab435493c8535ad9067bf595c" candidate_uuid: "efaf5e68ae1a470997f14d5b31ff7315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6809b102b684170b4282dc608d6cda0"
I20260806 01:07:28.343241 30159 raft_consensus.cc:3037] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:28.343261 29975 raft_consensus.cc:3037] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:28.345733 29975 raft_consensus.cc:2454] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efaf5e68ae1a470997f14d5b31ff7315 in term 1.
I20260806 01:07:28.345732 30159 raft_consensus.cc:2454] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efaf5e68ae1a470997f14d5b31ff7315 in term 1.
I20260806 01:07:28.346206 29699 leader_election.cc:302] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6809b102b684170b4282dc608d6cda0, efaf5e68ae1a470997f14d5b31ff7315; no voters:
I20260806 01:07:28.346558 30260 raft_consensus.cc:2781] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:28.347039 30260 raft_consensus.cc:686] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 1 LEADER]: Becoming Leader. State: Replica: efaf5e68ae1a470997f14d5b31ff7315, State: Running, Role: LEADER
I20260806 01:07:28.347536 30260 consensus_queue.cc:237] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } }
I20260806 01:07:28.352278 29594 catalog_manager.cc:5495] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 reported cstate change: term changed from 0 to 1, leader changed from <none> to efaf5e68ae1a470997f14d5b31ff7315 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "efaf5e68ae1a470997f14d5b31ff7315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efaf5e68ae1a470997f14d5b31ff7315" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:28.385316 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.383077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58082 (local address 127.22.63.65:40377)
0806 01:07:28.383289 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.383296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.383320 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:07:28.383510 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.383515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.383599 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.383720 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.384667 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.384671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.384674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.384690 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.384723 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.384735 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.384862 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.384965 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.385038 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:07:28.394027 29975 raft_consensus.cc:1261] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 1 FOLLOWER]: Refusing update from remote peer efaf5e68ae1a470997f14d5b31ff7315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:28.394027 30159 raft_consensus.cc:1261] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 1 FOLLOWER]: Refusing update from remote peer efaf5e68ae1a470997f14d5b31ff7315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:28.395049 30260 consensus_queue.cc:1035] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6809b102b684170b4282dc608d6cda0" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:28.395413 30265 consensus_queue.cc:1035] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "911931f0e74444ab9fb2a9faaf1dbd7f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:28.414345 30270 mvcc.cc:205] Tried to move back new op lower bound from 7315367724610965504 to 7315367724435857408. Current Snapshot: MvccSnapshot[applied={T|T < 7315367724610965504}]
I20260806 01:07:28.425065 29593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:28.425338 29593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:28.429023 29593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6dfef68792044f68d45ec0e21de2d2c: Sending DeleteTablet for 3 replicas of tablet 820f685ab435493c8535ad9067bf595c
I20260806 01:07:28.430285 29767 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f685ab435493c8535ad9067bf595c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:28 UTC) from {username='slave'} at 127.0.0.1:58072
I20260806 01:07:28.430333 30127 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f685ab435493c8535ad9067bf595c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:28 UTC) from {username='slave'} at 127.0.0.1:38762
I20260806 01:07:28.431322 29944 tablet_service.cc:1514] Processing DeleteTablet for tablet 820f685ab435493c8535ad9067bf595c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:28 UTC) from {username='slave'} at 127.0.0.1:45836
I20260806 01:07:28.431589 30283 tablet_replica.cc:330] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: stopping tablet replica
I20260806 01:07:28.431984 30284 tablet_replica.cc:330] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: stopping tablet replica
I20260806 01:07:28.432173 30283 raft_consensus.cc:2229] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:28.432278 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 29673
I20260806 01:07:28.432580 30284 raft_consensus.cc:2229] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:28.432832 30283 raft_consensus.cc:2258] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:28.433422 30284 raft_consensus.cc:2258] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:28.436574 30284 ts_tablet_manager.cc:1905] T 820f685ab435493c8535ad9067bf595c P efaf5e68ae1a470997f14d5b31ff7315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:28.440809 30286 tablet_replica.cc:330] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: stopping tablet replica
I20260806 01:07:28.441187 30283 ts_tablet_manager.cc:1905] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:28.441277 30286 raft_consensus.cc:2229] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:28.441820 30286 raft_consensus.cc:2258] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:28.443189 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.434078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.22.63.65:40377)
0806 01:07:28.434255 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.434262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.440413 (+ 6151us) server_negotiation.cc:407] Connection header received
0806 01:07:28.440803 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.440810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.440884 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.440995 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.442231 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.442237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.442241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.442263 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.442303 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.442317 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.442416 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.442670 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.442739 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:07:28.444353 29848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:28.446671 30286 ts_tablet_manager.cc:1905] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:28.447644 30283 ts_tablet_manager.cc:1918] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:28.447871 30283 log.cc:1196] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/820f685ab435493c8535ad9067bf595c
I20260806 01:07:28.448374 30283 ts_tablet_manager.cc:1939] T 820f685ab435493c8535ad9067bf595c P 911931f0e74444ab9fb2a9faaf1dbd7f: Deleting consensus metadata
I20260806 01:07:28.450080 29572 catalog_manager.cc:4841] TS 911931f0e74444ab9fb2a9faaf1dbd7f (127.22.63.67:46589): tablet 820f685ab435493c8535ad9067bf595c (table test-table [id=0d9a42ab7e0c47b4802ebfbb25944d1e]) successfully deleted
I20260806 01:07:28.454828 30286 ts_tablet_manager.cc:1918] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:28.454979 30286 log.cc:1196] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/820f685ab435493c8535ad9067bf595c
I20260806 01:07:28.455459 30286 ts_tablet_manager.cc:1939] T 820f685ab435493c8535ad9067bf595c P a6809b102b684170b4282dc608d6cda0: Deleting consensus metadata
I20260806 01:07:28.457178 29572 catalog_manager.cc:4841] TS a6809b102b684170b4282dc608d6cda0 (127.22.63.66:43481): tablet 820f685ab435493c8535ad9067bf595c (table test-table [id=0d9a42ab7e0c47b4802ebfbb25944d1e]) successfully deleted
I20260806 01:07:28.543346 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 29673
W20260806 01:07:28.572214 29570 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:40377 (error 108)
I20260806 01:07:28.573077 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 29861
I20260806 01:07:28.574555 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.573895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58102)
0806 01:07:28.574199 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.574372 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260806 01:07:28.574817 29570 catalog_manager.cc:4584] TS efaf5e68ae1a470997f14d5b31ff7315 (127.22.63.65:40377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 820f685ab435493c8535ad9067bf595c: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
I20260806 01:07:28.576231 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.573924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45838 (local address 127.22.63.66:43481)
0806 01:07:28.574136 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.574143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.574166 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:07:28.574325 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.574332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.574457 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.574568 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.575640 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.575644 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.575647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.575663 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.575693 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.575704 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.575887 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.575974 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.576021 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:07:28.576876 30034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:28.628846 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.628353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58104)
0806 01:07:28.628573 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.628678 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:07:28.629925 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.629591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58116)
0806 01:07:28.629742 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.629813 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:07:28.666210 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 29861
I20260806 01:07:28.690626 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.689998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58124)
0806 01:07:28.690260 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.690412 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:07:28.691715 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.691362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58128)
0806 01:07:28.691516 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.691597 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 01:07:28.692545 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30044
I20260806 01:07:28.695387 30247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.693321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.22.63.67:46589)
0806 01:07:28.693544 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:28.693553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:28.693572 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:07:28.693637 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:28.693640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:28.693718 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:28.693826 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:28.694738 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:28.694744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:28.694749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:28.694766 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:28.694807 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:28.694819 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:28.694956 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:28.695048 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 01:07:28.695112 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:07:28.696260 30219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:28.772886 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.772407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58140)
0806 01:07:28.772628 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.772730 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:07:28.773985 30249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:28.773553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40377 (local address 127.0.0.1:58156)
0806 01:07:28.773771 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:28.773839 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:40377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 01:07:28.787899 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30044
I20260806 01:07:28.813424 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 29549
I20260806 01:07:28.814651 29635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:28.922695 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 29549
2026-08-06T01:07:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2695 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 01:07:28.975379 22781 mini_oidc.cc:152] Starting JWKS server
I20260806 01:07:28.978340 22781 webserver.cc:460] Webserver started at https://127.0.0.1:34669/ using document root <none> and password file <none>
I20260806 01:07:28.978679 22781 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:07:28.979333 22781 webserver.cc:460] Webserver started at http://127.0.0.1:37879/ using document root <none> and password file <none>
2026-08-06T01:07:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:28Z Disabled control of system clock
I20260806 01:07:29.014974 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40029
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40029
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/testchainca.cert with env {}
W20260806 01:07:29.350637 30306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:29.351003 30306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:29.351174 30306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:29.358557 30306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:29.358712 30306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:29.358805 30306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:29.358918 30306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:29.359006 30306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:07:29.359100 30306 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:07:29.359181 30306 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34669/jwks.json
I20260806 01:07:29.369328 30306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:29.370709 30306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:29.372124 30306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:29.384634 30312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:29.384544 30319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:29.384603 30315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:29.385780 30306 server_base.cc:1034] running on GCE node
I20260806 01:07:29.386586 30306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:29.388094 30306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:29.389302 30306 hybrid_clock.cc:648] HybridClock initialized: now 1785978449389250 us; error 48 us; skew 500 ppm
I20260806 01:07:29.389693 30306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:29.392151 30306 webserver.cc:460] Webserver started at http://127.22.63.126:45757/ using document root <none> and password file <none>
I20260806 01:07:29.392887 30306 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:29.393013 30306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:29.393355 30306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:29.396409 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "8fb9f3550a1c475190f5b07cb76aa049"
format_stamp: "Formatted at 2026-08-06 01:07:29 on dist-test-slave-7f01"
server_key: "664c956552c88754631c089d71e89b5a"
server_key_iv: "a5586d34c2c851e063741166a1ee8422"
server_key_version: "encryptionkey@0"
I20260806 01:07:29.397228 30306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "8fb9f3550a1c475190f5b07cb76aa049"
format_stamp: "Formatted at 2026-08-06 01:07:29 on dist-test-slave-7f01"
server_key: "664c956552c88754631c089d71e89b5a"
server_key_iv: "a5586d34c2c851e063741166a1ee8422"
server_key_version: "encryptionkey@0"
I20260806 01:07:29.402413 30306 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:29.405938 30329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:29.407334 30306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:29.407548 30306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "8fb9f3550a1c475190f5b07cb76aa049"
format_stamp: "Formatted at 2026-08-06 01:07:29 on dist-test-slave-7f01"
server_key: "664c956552c88754631c089d71e89b5a"
server_key_iv: "a5586d34c2c851e063741166a1ee8422"
server_key_version: "encryptionkey@0"
I20260806 01:07:29.407789 30306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:29.425953 30306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:29.426829 30306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:29.427156 30306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:29.446578 30306 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40029
I20260806 01:07:29.446640 30403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40029 every 8 connection(s)
I20260806 01:07:29.448113 30306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:29.452098 30404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:29.457206 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30306
I20260806 01:07:29.457472 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:29.458290 22781 external_mini_cluster.cc:1428] Setting key 4c66bf4f78e2ad7e493622b75bc2b170
I20260806 01:07:29.463953 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049: Bootstrap starting.
I20260806 01:07:29.467641 30404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:29.468845 30404 log.cc:824] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:29.471987 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049: No bootstrap required, opened a new log
I20260806 01:07:29.472793 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:29.460373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.22.63.126:40029)
0806 01:07:29.461534 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:29.461546 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:29.461590 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:07:29.463338 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:29.463361 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:29.463879 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:29.464234 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:29.464882 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.466161 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.468148 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.468831 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.469090 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:29.470414 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:29.470422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:29.470437 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:29.470965 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:29.471041 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:29.471062 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:29.471155 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:29.471404 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 01:07:29.471778 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":505,"threads_started":1}
I20260806 01:07:29.476617 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:07:29.481141 30404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } }
I20260806 01:07:29.481501 30404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:29.481576 30404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb9f3550a1c475190f5b07cb76aa049, State: Initialized, Role: FOLLOWER
I20260806 01:07:29.482060 30404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } }
I20260806 01:07:29.482246 30404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:29.482343 30404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:29.482482 30404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:29.484686 30404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } }
I20260806 01:07:29.485142 30404 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fb9f3550a1c475190f5b07cb76aa049; no voters:
I20260806 01:07:29.485740 30404 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:29.485913 30412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:29.487187 30412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [term 1 LEADER]: Becoming Leader. State: Replica: 8fb9f3550a1c475190f5b07cb76aa049, State: Running, Role: LEADER
I20260806 01:07:29.487816 30412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } }
I20260806 01:07:29.488260 30404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:29.490906 30414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fb9f3550a1c475190f5b07cb76aa049. Latest consensus state: current_term: 1 leader_uuid: "8fb9f3550a1c475190f5b07cb76aa049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } } }
I20260806 01:07:29.490908 30413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fb9f3550a1c475190f5b07cb76aa049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fb9f3550a1c475190f5b07cb76aa049" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40029 } } }
I20260806 01:07:29.491379 30414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:29.491426 30413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:29.494163 30427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:29.500597 30427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:29.509809 30427 catalog_manager.cc:1347] Generated new cluster ID: 483b0f812fff48eca74d2ef08f451da8
I20260806 01:07:29.509953 30427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:29.529815 30427 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:29.531106 30427 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:29.542660 30427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fb9f3550a1c475190f5b07cb76aa049: Generated new TSK 0
I20260806 01:07:29.543531 30427 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:07:29.588629 30450 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 01:07:29.588838 30450 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:07:29.590137 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:29.560709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.22.63.126:40029)
0806 01:07:29.560962 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:29.560982 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:29.561464 (+ 482us) server_negotiation.cc:407] Connection header received
0806 01:07:29.561888 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:29.561892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:29.561970 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:29.562123 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:07:29.562712 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.563475 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.565022 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.565489 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.565632 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:29.566215 (+ 583us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:07:29.589354 (+ 23139us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:07:29.589795 (+ 441us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 01:07:29.590286 30407 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 01:07:29.592033 30448 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:29.560671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40029 (local address 127.0.0.1:47340)
0806 01:07:29.561350 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:29.561366 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:29.561571 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:29.562161 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:29.562166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:29.562180 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:07:29.562549 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:29.562558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.563659 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.563664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:29.564875 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:29.564887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:29.565689 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:29.565698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:29.565846 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:29.565907 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:07:29.589713 (+ 23806us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 01:07:29.590097 (+ 384us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.63.126:40029: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":363,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.63.126:40029: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 01:07:29.593930 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30306
I20260806 01:07:29.595192 30399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:29.692288 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30306
2026-08-06T01:07:29Z chronyd exiting
I20260806 01:07:29.753435 22781 test_util.cc:175] -----------------------------------------------
I20260806 01:07:29.753587 22781 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785978402713339-22781-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (782 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 01:07:29.757691 22781 mini_oidc.cc:152] Starting JWKS server
I20260806 01:07:29.759603 22781 webserver.cc:460] Webserver started at https://127.0.0.1:35403/ using document root <none> and password file <none>
I20260806 01:07:29.759891 22781 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:07:29.760586 22781 webserver.cc:460] Webserver started at http://127.0.0.1:36561/ using document root <none> and password file <none>
2026-08-06T01:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:29Z Disabled control of system clock
I20260806 01:07:29.797377 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:44711
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:44711
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35403/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/testchainca.cert with env {}
W20260806 01:07:30.140534 30458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:30.140872 30458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:30.141049 30458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:30.148435 30458 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:30.148586 30458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:30.148676 30458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:30.148784 30458 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:30.148875 30458 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:07:30.148972 30458 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:07:30.149049 30458 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35403/jwks.json
I20260806 01:07:30.159938 30458 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:44711
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:44711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35403/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:30.161422 30458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:30.162832 30458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:30.174839 30464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:30.174888 30467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:30.174796 30465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:30.176307 30458 server_base.cc:1034] running on GCE node
I20260806 01:07:30.177217 30458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:30.178683 30458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:30.179922 30458 hybrid_clock.cc:648] HybridClock initialized: now 1785978450179887 us; error 29 us; skew 500 ppm
I20260806 01:07:30.180284 30458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:30.182601 30458 webserver.cc:460] Webserver started at http://127.22.63.126:39063/ using document root <none> and password file <none>
I20260806 01:07:30.183348 30458 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:30.183497 30458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:30.183913 30458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:30.186801 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "f5f23d059d98436ea6194dbfdcf89658"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "8daccbfbe591107df9ad70d7e0fe2c31"
server_key_iv: "e2e829ce860264877c408b8a9f76a79e"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.187623 30458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "f5f23d059d98436ea6194dbfdcf89658"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "8daccbfbe591107df9ad70d7e0fe2c31"
server_key_iv: "e2e829ce860264877c408b8a9f76a79e"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.192813 30458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:07:30.196528 30478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:30.197898 30458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:30.198113 30458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "f5f23d059d98436ea6194dbfdcf89658"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "8daccbfbe591107df9ad70d7e0fe2c31"
server_key_iv: "e2e829ce860264877c408b8a9f76a79e"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.198359 30458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:30.234329 30458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:30.235311 30458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:30.235635 30458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:30.255033 30458 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:44711
I20260806 01:07:30.255069 30550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:44711 every 8 connection(s)
I20260806 01:07:30.256453 30458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:30.260249 30552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:30.265316 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30458
I20260806 01:07:30.265522 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:30.266335 22781 external_mini_cluster.cc:1428] Setting key a786e1d1cfbb3a57d3875afdcad4061b
I20260806 01:07:30.271654 30552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658: Bootstrap starting.
I20260806 01:07:30.275310 30552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:30.276489 30552 log.cc:824] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:30.279599 30552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658: No bootstrap required, opened a new log
I20260806 01:07:30.280453 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:30.268402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.22.63.126:44711)
0806 01:07:30.269536 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:30.269547 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:30.269592 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:07:30.271216 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:30.271237 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:30.271681 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:30.272078 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:30.272691 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.274010 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.275864 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.276514 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.276711 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:30.278063 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:30.278070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:30.278084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:30.278615 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:30.278688 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:30.278709 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:30.278795 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:30.279051 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 01:07:30.279385 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":462,"threads_started":1}
I20260806 01:07:30.284359 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:07:30.287624 30552 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } }
I20260806 01:07:30.288166 30552 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:30.288290 30552 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f23d059d98436ea6194dbfdcf89658, State: Initialized, Role: FOLLOWER
I20260806 01:07:30.289086 30552 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } }
I20260806 01:07:30.289362 30562 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:30.289614 30562 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } }
I20260806 01:07:30.290210 30552 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:30.290349 30552 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:30.290211 30562 leader_election.cc:302] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5f23d059d98436ea6194dbfdcf89658; no voters:
I20260806 01:07:30.290555 30552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:30.291193 30562 leader_election.cc:288] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260806 01:07:30.293367 30552 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } }
I20260806 01:07:30.293627 30552 leader_election.cc:302] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5f23d059d98436ea6194dbfdcf89658; no voters:
I20260806 01:07:30.293685 30563 raft_consensus.cc:2741] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 01:07:30.293880 30552 leader_election.cc:288] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:30.294073 30562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:30.296383 30552 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:30.296789 30562 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [term 1 LEADER]: Becoming Leader. State: Replica: f5f23d059d98436ea6194dbfdcf89658, State: Running, Role: LEADER
I20260806 01:07:30.297375 30562 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } }
I20260806 01:07:30.300021 30564 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5f23d059d98436ea6194dbfdcf89658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } } }
I20260806 01:07:30.300055 30563 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5f23d059d98436ea6194dbfdcf89658. Latest consensus state: current_term: 1 leader_uuid: "f5f23d059d98436ea6194dbfdcf89658" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5f23d059d98436ea6194dbfdcf89658" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 44711 } } }
I20260806 01:07:30.300450 30564 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:30.300519 30563 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:30.301438 30578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:30.307545 30578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:30.316613 30578 catalog_manager.cc:1347] Generated new cluster ID: ffdc1670ddf6418cb83774178c0bd247
I20260806 01:07:30.316735 30578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:30.333796 30578 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:30.335649 30578 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:30.352300 30578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5f23d059d98436ea6194dbfdcf89658: Generated new TSK 0
I20260806 01:07:30.353392 30578 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 01:07:30.404577 30607 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 01:07:30.404713 30607 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:07:30.406031 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:30.379029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.22.63.126:44711)
0806 01:07:30.381727 (+ 2698us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:30.381734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:30.382331 (+ 597us) server_negotiation.cc:407] Connection header received
0806 01:07:30.382635 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:30.382645 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:30.382776 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:30.382989 (+ 213us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:07:30.383639 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.384870 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.386883 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.387771 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.387901 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:30.388504 (+ 603us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:07:30.405300 (+ 16796us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:07:30.405721 (+ 421us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":2547}
W20260806 01:07:30.406165 30557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51340: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 01:07:30.406263 30604 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:30.378942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:44711 (local address 127.0.0.1:51340)
0806 01:07:30.382195 (+ 3253us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:30.382216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:30.382389 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:30.383077 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:30.383082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:30.383100 (+ 18us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:07:30.383449 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:30.383462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.385030 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.385037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:30.386722 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:30.386738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:30.387978 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:30.387986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:30.388135 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:30.388171 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:07:30.405564 (+ 17393us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 01:07:30.405845 (+ 281us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.63.126:44711: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35403/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":3092,"thread_start_us":2940,"threads_started":1}
I20260806 01:07:30.407459 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30458
I20260806 01:07:30.408748 30543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:30.507251 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30458
2026-08-06T01:07:30Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (811 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 01:07:30.569615 22781 mini_oidc.cc:152] Starting JWKS server
I20260806 01:07:30.571563 22781 webserver.cc:460] Webserver started at https://127.0.0.1:44395/ using document root <none> and password file <none>
I20260806 01:07:30.571848 22781 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 01:07:30.572512 22781 webserver.cc:460] Webserver started at http://127.0.0.1:42555/ using document root <none> and password file <none>
2026-08-06T01:07:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:30Z Disabled control of system clock
I20260806 01:07:30.608393 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33885
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33885
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44395/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:07:30.941077 30615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:30.941382 30615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:30.941473 30615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:30.948684 30615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:30.948799 30615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:30.948836 30615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:30.948900 30615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:30.948945 30615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:07:30.949002 30615 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:07:30.949043 30615 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44395/jwks.json
I20260806 01:07:30.958168 30615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44395/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:30.959569 30615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:30.961457 30615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:30.973871 30621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:30.973984 30622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:30.973989 30625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:30.975324 30615 server_base.cc:1034] running on GCE node
I20260806 01:07:30.976219 30615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:30.977710 30615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:30.978976 30615 hybrid_clock.cc:648] HybridClock initialized: now 1785978450978927 us; error 55 us; skew 500 ppm
I20260806 01:07:30.979331 30615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:30.981788 30615 webserver.cc:460] Webserver started at http://127.22.63.126:34155/ using document root <none> and password file <none>
I20260806 01:07:30.982509 30615 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:30.982645 30615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:30.983039 30615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:30.986059 30615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "77d3c626f1cc4960821aea494aed4849"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "15df7f43c93fc32b997f1179ada1c4f2"
server_key_iv: "49796074a10776aa6f3ed05909b39f27"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.986904 30615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "77d3c626f1cc4960821aea494aed4849"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "15df7f43c93fc32b997f1179ada1c4f2"
server_key_iv: "49796074a10776aa6f3ed05909b39f27"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.992177 30615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:07:30.995615 30631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:30.997011 30615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:30.997207 30615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "77d3c626f1cc4960821aea494aed4849"
format_stamp: "Formatted at 2026-08-06 01:07:30 on dist-test-slave-7f01"
server_key: "15df7f43c93fc32b997f1179ada1c4f2"
server_key_iv: "49796074a10776aa6f3ed05909b39f27"
server_key_version: "encryptionkey@0"
I20260806 01:07:30.997427 30615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:31.023048 30615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:31.023974 30615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:31.024305 30615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:31.042786 30615 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33885
I20260806 01:07:31.042807 30704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33885 every 8 connection(s)
I20260806 01:07:31.044234 30615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:31.048074 30705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:31.051782 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30615
I20260806 01:07:31.052009 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:31.052664 22781 external_mini_cluster.cc:1428] Setting key 3ff55569e315e901b3553b53878beed8
I20260806 01:07:31.059115 30705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849: Bootstrap starting.
I20260806 01:07:31.062449 30705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:31.063508 30705 log.cc:824] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:31.066385 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:31.054589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.22.63.126:33885)
0806 01:07:31.055522 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:31.055532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:31.055577 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:07:31.057835 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:31.057857 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:31.058307 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:31.058673 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:31.059275 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.060685 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.062016 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.062731 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.062968 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.064157 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:31.064168 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:31.064182 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:31.064766 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:31.064828 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:31.064848 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:31.064990 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:31.065194 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 01:07:31.065493 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":315,"threads_started":1}
I20260806 01:07:31.066725 30705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849: No bootstrap required, opened a new log
I20260806 01:07:31.070333 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:07:31.075142 30705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } }
I20260806 01:07:31.075534 30705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:31.075668 30705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d3c626f1cc4960821aea494aed4849, State: Initialized, Role: FOLLOWER
I20260806 01:07:31.076301 30705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } }
I20260806 01:07:31.076546 30705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:31.076718 30705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:31.076925 30705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:31.079348 30705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } }
I20260806 01:07:31.080036 30705 leader_election.cc:302] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77d3c626f1cc4960821aea494aed4849; no voters:
I20260806 01:07:31.080694 30705 leader_election.cc:288] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:31.080904 30710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:31.081637 30710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [term 1 LEADER]: Becoming Leader. State: Replica: 77d3c626f1cc4960821aea494aed4849, State: Running, Role: LEADER
I20260806 01:07:31.082213 30710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } }
I20260806 01:07:31.083068 30705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:31.085075 30712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77d3c626f1cc4960821aea494aed4849. Latest consensus state: current_term: 1 leader_uuid: "77d3c626f1cc4960821aea494aed4849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } } }
I20260806 01:07:31.085597 30712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:31.087009 30711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77d3c626f1cc4960821aea494aed4849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77d3c626f1cc4960821aea494aed4849" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33885 } } }
I20260806 01:07:31.087431 30711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:31.088778 30722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:31.094285 30722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:31.103565 30722 catalog_manager.cc:1347] Generated new cluster ID: 578e0fbeb4d64d93b64f49bfedc3da94
I20260806 01:07:31.103771 30722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:31.121356 30722 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:31.122602 30722 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:31.133067 30722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77d3c626f1cc4960821aea494aed4849: Generated new TSK 0
I20260806 01:07:31.133940 30722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:07:31.157212 30708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:31.148663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49262 (local address 127.22.63.126:33885)
0806 01:07:31.148864 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:31.148871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:31.149258 (+ 387us) server_negotiation.cc:407] Connection header received
0806 01:07:31.149473 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:31.149479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:31.149575 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:31.149737 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:07:31.150207 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.151335 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.152759 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.153469 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.153608 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.154148 (+ 540us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:07:31.156557 (+ 2409us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:07:31.156889 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 01:07:31.157346 30708 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49262: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 01:07:31.158972 30743 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:31.148532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:33885 (local address 127.0.0.1:49262)
0806 01:07:31.149151 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:31.149171 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:31.149290 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:31.149761 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:31.149765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:31.149775 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:07:31.150054 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:31.150061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.151487 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.151491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:31.152624 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:31.152635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.153683 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.153688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:31.153817 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.153853 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:07:31.156769 (+ 2916us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 01:07:31.158540 (+ 1771us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.63.126:33885: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":1474,"thread_start_us":359,"threads_started":1}
I20260806 01:07:31.160049 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30615
I20260806 01:07:31.161142 30699 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 01:07:31.172511 30744 webserver.cc:533] Webserver: sslize failed:
E20260806 01:07:31.172649 30744 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 01:07:31.255071 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30615
2026-08-06T01:07:31Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (747 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T01:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:31Z Disabled control of system clock
I20260806 01:07:31.354943 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43445
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:46499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:43445
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/testchainca.cert with env {}
W20260806 01:07:31.692883 30751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:31.693202 30751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:31.693365 30751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:31.700865 30751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:31.700984 30751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:31.701081 30751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:31.701176 30751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:31.701227 30751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 01:07:31.701356 30751 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 01:07:31.701418 30751 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 01:07:31.712005 30751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:43445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:31.713397 30751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:31.714761 30751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:31.727527 30759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:31.727456 30762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:31.728102 30758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:31.728394 30751 server_base.cc:1034] running on GCE node
I20260806 01:07:31.729233 30751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:31.730710 30751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:31.731927 30751 hybrid_clock.cc:648] HybridClock initialized: now 1785978451731891 us; error 56 us; skew 500 ppm
I20260806 01:07:31.732287 30751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:07:31.737679 30751 webserver.cc:460] Webserver started at http://127.22.63.126:37843/ using document root <none> and password file <none>
I20260806 01:07:31.738513 30751 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:31.738636 30751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:31.738976 30751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:31.742023 30751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "4bac1e2c37254d0db0594f5689511ebe"
format_stamp: "Formatted at 2026-08-06 01:07:31 on dist-test-slave-7f01"
server_key: "994d18917be5bdc7add63c7e629970fb"
server_key_iv: "892b9b4149d0d5989c2a83ff47acfdfa"
server_key_version: "encryptionkey@0"
I20260806 01:07:31.742849 30751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "4bac1e2c37254d0db0594f5689511ebe"
format_stamp: "Formatted at 2026-08-06 01:07:31 on dist-test-slave-7f01"
server_key: "994d18917be5bdc7add63c7e629970fb"
server_key_iv: "892b9b4149d0d5989c2a83ff47acfdfa"
server_key_version: "encryptionkey@0"
I20260806 01:07:31.748157 30751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:07:31.751511 30771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:31.752859 30751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:31.753055 30751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "4bac1e2c37254d0db0594f5689511ebe"
format_stamp: "Formatted at 2026-08-06 01:07:31 on dist-test-slave-7f01"
server_key: "994d18917be5bdc7add63c7e629970fb"
server_key_iv: "892b9b4149d0d5989c2a83ff47acfdfa"
server_key_version: "encryptionkey@0"
I20260806 01:07:31.753278 30751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:31.779814 30751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:31.780707 30751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:31.781018 30751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:31.800323 30751 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:43445
I20260806 01:07:31.800346 30850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:43445 every 8 connection(s)
I20260806 01:07:31.801721 30751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:31.805621 30851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:31.807156 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30751
I20260806 01:07:31.807368 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:31.808117 22781 external_mini_cluster.cc:1428] Setting key b36732bb51cf97ed87fc165448b35ad1
I20260806 01:07:31.817579 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe: Bootstrap starting.
I20260806 01:07:31.821341 30851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:31.822142 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:31.810160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.22.63.126:43445)
0806 01:07:31.811197 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:31.811208 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:31.811253 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:07:31.812932 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:31.812954 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:31.813391 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:31.813717 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:31.814327 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.815676 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.817505 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.818235 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.818380 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.819678 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:31.819685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:31.819699 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:07:31.820307 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:31.820393 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:07:31.820417 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:31.820513 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:31.820764 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 01:07:31.821120 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":372,"threads_started":1}
I20260806 01:07:31.822636 30851 log.cc:824] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:31.826046 30851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe: No bootstrap required, opened a new log
I20260806 01:07:31.826601 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:07:31.834227 30851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } }
I20260806 01:07:31.834656 30851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:31.834757 30851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bac1e2c37254d0db0594f5689511ebe, State: Initialized, Role: FOLLOWER
I20260806 01:07:31.835346 30851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } }
I20260806 01:07:31.835559 30851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:31.835726 30851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:31.835919 30851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:31.838474 30851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } }
I20260806 01:07:31.839030 30851 leader_election.cc:302] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bac1e2c37254d0db0594f5689511ebe; no voters:
I20260806 01:07:31.839613 30851 leader_election.cc:288] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:31.839802 30857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:31.840540 30857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [term 1 LEADER]: Becoming Leader. State: Replica: 4bac1e2c37254d0db0594f5689511ebe, State: Running, Role: LEADER
I20260806 01:07:31.841080 30857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } }
I20260806 01:07:31.841883 30851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:31.843806 30859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bac1e2c37254d0db0594f5689511ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } } }
I20260806 01:07:31.844878 30859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:31.844460 30860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bac1e2c37254d0db0594f5689511ebe. Latest consensus state: current_term: 1 leader_uuid: "4bac1e2c37254d0db0594f5689511ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bac1e2c37254d0db0594f5689511ebe" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43445 } } }
I20260806 01:07:31.845240 30860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:31.850064 30867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:31.855500 30867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:31.864631 30867 catalog_manager.cc:1347] Generated new cluster ID: 131db3bdddfa441b87666fd0e3eb0ae3
I20260806 01:07:31.864765 30867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:31.874444 30867 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:31.875838 30867 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:31.884435 30867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bac1e2c37254d0db0594f5689511ebe: Generated new TSK 0
I20260806 01:07:31.885525 30867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:07:32.330581 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:31.900091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.22.63.126:43445)
0806 01:07:31.900297 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:31.900303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:31.900790 (+ 487us) server_negotiation.cc:407] Connection header received
0806 01:07:31.901000 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:31.901006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:31.901105 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:31.901297 (+ 192us) server_negotiation.cc:226] Negotiated authn=JWT
0806 01:07:31.901706 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.902844 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.904342 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.905053 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.905189 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.905662 (+ 473us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 01:07:32.329977 (+424315us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:07:32.330339 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42362: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 01:07:32.330700 30855 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42362: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 01:07:32.331029 30891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:31.900045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43445 (local address 127.0.0.1:42362)
0806 01:07:31.900681 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:31.900697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:31.900830 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:31.901307 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:31.901310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:31.901323 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0806 01:07:31.901560 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:31.901567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.903029 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.903033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:31.904200 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:31.904212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:31.905235 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:31.905239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:31.905364 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:31.905397 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 01:07:32.330296 (+424899us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 01:07:32.330572 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.63.126:43445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":347,"threads_started":1}
I20260806 01:07:32.332453 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30751
I20260806 01:07:32.333581 30842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:32.438994 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30751
2026-08-06T01:07:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1165 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (433 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (435 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:33 dist-test-slave-7f01 krb5kdc[30912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:33 dist-test-slave-7f01 krb5kdc[30912](info): set up 2 sockets
Aug 06 01:07:33 dist-test-slave-7f01 krb5kdc[30912](info): commencing operation
krb5kdc: starting...
W20260806 01:07:34.610657 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:34 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:34.865803 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40859
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:35.207661 30932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:35.208030 30932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:35.208187 30932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:35.215423 30932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:35.215525 30932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:35.215633 30932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:35.215734 30932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:35.215844 30932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:35.225672 30932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:35.227016 30932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:35.228447 30932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:35.240465 30941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:35.240547 30943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:35.240427 30945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:35.241819 30932 server_base.cc:1034] running on GCE node
I20260806 01:07:35.242538 30932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:35.243934 30932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:35.245148 30932 hybrid_clock.cc:648] HybridClock initialized: now 1785978455245106 us; error 36 us; skew 500 ppm
I20260806 01:07:35.245446 30932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:35 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978455, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:35.248718 30932 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:07:35.250232 30932 webserver.cc:460] Webserver started at http://127.22.63.126:37643/ using document root <none> and password file <none>
I20260806 01:07:35.250946 30932 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:35.251081 30932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:35.251434 30932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:35.254575 30932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "f5e0797fbf604ca0b1a439e20b923021"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "709ce0451ff954ee7897fc007ec68b14"
server_key_iv: "dafe60a7d65885f4182e39e3d5486b5b"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.255398 30932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "f5e0797fbf604ca0b1a439e20b923021"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "709ce0451ff954ee7897fc007ec68b14"
server_key_iv: "dafe60a7d65885f4182e39e3d5486b5b"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.261011 30932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:07:35.264635 30953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:35.266100 30932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:35.266295 30932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "f5e0797fbf604ca0b1a439e20b923021"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "709ce0451ff954ee7897fc007ec68b14"
server_key_iv: "dafe60a7d65885f4182e39e3d5486b5b"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.266512 30932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:35.284914 30932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:35.289170 30932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:35.289592 30932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:35.308637 30932 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40859
I20260806 01:07:35.308676 31034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40859 every 8 connection(s)
I20260806 01:07:35.310068 30932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:35.313100 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 30932
I20260806 01:07:35.313333 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:35.314189 22781 external_mini_cluster.cc:1428] Setting key 5ab6ca6f35d37ec452bdd62a54eca13e
I20260806 01:07:35.314334 31035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:35.329304 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021: Bootstrap starting.
Aug 06 01:07:35 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:35.333319 31035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:35.334654 31035 log.cc:824] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:35.338212 31035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021: No bootstrap required, opened a new log
I20260806 01:07:35.339454 31039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:35.316385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.22.63.126:40859)
0806 01:07:35.317188 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:35.317200 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:35.317245 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:07:35.319123 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:35.319145 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:35.319542 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:35.319950 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:35.321731 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.323093 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.324914 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.325636 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.325790 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:35.330617 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:35.330627 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:35.330647 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:35.330698 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:35.334784 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:35.335609 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:35.335615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:35.335624 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:35.335853 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:35.336301 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:35.336305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:35.336308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:35.336999 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:35.337306 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:35.337669 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 01:07:35.338081 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":384,"threads_started":1}
I20260806 01:07:35.346797 31035 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } }
I20260806 01:07:35.347157 31035 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:35.347227 31035 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5e0797fbf604ca0b1a439e20b923021, State: Initialized, Role: FOLLOWER
I20260806 01:07:35.347823 31035 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } }
I20260806 01:07:35.348068 31035 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:35.348234 31035 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:35.348431 31035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:35.350714 31035 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } }
I20260806 01:07:35.351154 31035 leader_election.cc:302] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5e0797fbf604ca0b1a439e20b923021; no voters:
I20260806 01:07:35.351682 31035 leader_election.cc:288] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:35.351882 31042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:35.352600 31042 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [term 1 LEADER]: Becoming Leader. State: Replica: f5e0797fbf604ca0b1a439e20b923021, State: Running, Role: LEADER
I20260806 01:07:35.353188 31042 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } }
I20260806 01:07:35.353801 31035 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:35.357946 31044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5e0797fbf604ca0b1a439e20b923021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } } }
I20260806 01:07:35.358135 31045 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5e0797fbf604ca0b1a439e20b923021. Latest consensus state: current_term: 1 leader_uuid: "f5e0797fbf604ca0b1a439e20b923021" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5e0797fbf604ca0b1a439e20b923021" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40859 } } }
I20260806 01:07:35.358354 31044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:35.358389 31045 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:35.359364 31060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:35.364254 31060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:35.373047 31060 catalog_manager.cc:1347] Generated new cluster ID: fa80836cc0d0431384862d8a9ad5b858
I20260806 01:07:35.373167 31060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:35.389518 31060 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:35.390821 31060 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:35.403378 31060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5e0797fbf604ca0b1a439e20b923021: Generated new TSK 0
I20260806 01:07:35.404291 31060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:07:35.467227 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:35.817118 31073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:35.817416 31073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:35.817576 31073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:35.825009 31073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:35.825124 31073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:35.825292 31073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:35.835392 31073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:35.836997 31073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:35.838299 31073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:35.852279 31079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:35.852360 31080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:35.852936 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:35.854144 31073 server_base.cc:1034] running on GCE node
I20260806 01:07:35.854909 31073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:35.856016 31073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:35.857264 31073 hybrid_clock.cc:648] HybridClock initialized: now 1785978455857221 us; error 39 us; skew 500 ppm
I20260806 01:07:35.857604 31073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:35 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978455, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:35.861299 31073 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:07:35.862908 31073 webserver.cc:460] Webserver started at http://127.22.63.65:35125/ using document root <none> and password file <none>
I20260806 01:07:35.863646 31073 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:35.863839 31073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:35.864198 31073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:35.867261 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "beb54773d0cb47d088fa063536b5a44a"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "c5ea8a1a72c2fdfccc1fb56a18ce148d"
server_key_iv: "50b27ba062fcf465ea5e44f98a4608a1"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.868191 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "beb54773d0cb47d088fa063536b5a44a"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "c5ea8a1a72c2fdfccc1fb56a18ce148d"
server_key_iv: "50b27ba062fcf465ea5e44f98a4608a1"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.873621 31073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 01:07:35.877759 31095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:35.879642 31073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:07:35.879889 31073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "beb54773d0cb47d088fa063536b5a44a"
format_stamp: "Formatted at 2026-08-06 01:07:35 on dist-test-slave-7f01"
server_key: "c5ea8a1a72c2fdfccc1fb56a18ce148d"
server_key_iv: "50b27ba062fcf465ea5e44f98a4608a1"
server_key_version: "encryptionkey@0"
I20260806 01:07:35.880172 31073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:35.912887 31073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:35.917289 31073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:35.917563 31073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:35.918557 31073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:35.919978 31073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:35.920054 31073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:35.920148 31073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:35.920194 31073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:35.956610 31073 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39543
I20260806 01:07:35.956664 31271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39543 every 8 connection(s)
I20260806 01:07:35.958070 31073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:35.964617 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 31073
I20260806 01:07:35.964877 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:35.965593 22781 external_mini_cluster.cc:1428] Setting key efc0a03058e8d7d6e6359f4032e43ea7
Aug 06 01:07:35 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978455, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:35.978611 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:35.960838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:50883 (local address 127.22.63.126:40859)
0806 01:07:35.961325 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:35.961330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:35.962231 (+ 901us) server_negotiation.cc:407] Connection header received
0806 01:07:35.964173 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:35.964179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:35.964280 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:35.964404 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:35.966104 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.966985 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.968733 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.969130 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.969269 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:35.972923 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:35.972928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:35.972931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:35.972965 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:35.975486 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:35.976287 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:35.976299 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:35.976302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:35.976420 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:35.976865 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:35.976870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:35.976880 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:35.977134 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:35.977292 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:35.977986 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 01:07:35.978226 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":263,"threads_started":1}
I20260806 01:07:35.979578 31277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:35.961317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40859 (local address 127.22.63.65:50883)
0806 01:07:35.962059 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:35.962100 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:35.963891 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:35.964657 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:35.964666 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:35.965302 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:35.965923 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:35.965942 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.967174 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.967185 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:35.968579 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:35.968593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:35.969385 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:35.969393 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:35.969698 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:35.970396 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:35.970420 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:35.972703 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:35.975642 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:35.975650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:35.975666 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:35.976155 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:35.976544 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:35.976550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:35.976554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:35.976742 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:35.977268 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:35.977276 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:35.977615 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 01:07:35.977876 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 01:07:35.978424 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":314,"threads_started":1}
I20260806 01:07:35.981340 31274 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40859
I20260806 01:07:35.981657 31274 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:35.982381 31274 heartbeater.cc:511] Master 127.22.63.126:40859 requested a full tablet report, sending...
I20260806 01:07:35.984910 30976 ts_manager.cc:194] Registered new tserver with Master: beb54773d0cb47d088fa063536b5a44a (127.22.63.65:39543)
I20260806 01:07:35.987329 30976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:50883
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:07:36.026674 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:36.368265 31282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:36.368568 31282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:36.368666 31282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:36.376015 31282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:36.376150 31282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:36.376379 31282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:36.387199 31282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:36.388638 31282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:36.390034 31282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:36.403646 31294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:36.403755 31293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:36.404258 31296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:36.404410 31282 server_base.cc:1034] running on GCE node
I20260806 01:07:36.405001 31282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:36.405956 31282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:36.407217 31282 hybrid_clock.cc:648] HybridClock initialized: now 1785978456407170 us; error 29 us; skew 500 ppm
I20260806 01:07:36.407600 31282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:36 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978456, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:36.411160 31282 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:07:36.412773 31282 webserver.cc:460] Webserver started at http://127.22.63.66:33861/ using document root <none> and password file <none>
I20260806 01:07:36.413552 31282 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:36.413717 31282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:36.414072 31282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:36.417397 31282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "8ca5c60b9c6e4d25937ba01794a83594"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "6edc9bcba15b2406f8ce6749ebb41ad7"
server_key_iv: "ec19de5486c09ec2eeaf18b43864258f"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.418249 31282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "8ca5c60b9c6e4d25937ba01794a83594"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "6edc9bcba15b2406f8ce6749ebb41ad7"
server_key_iv: "ec19de5486c09ec2eeaf18b43864258f"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.423956 31282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:07:36.427680 31304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:36.429145 31282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:36.429352 31282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "8ca5c60b9c6e4d25937ba01794a83594"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "6edc9bcba15b2406f8ce6749ebb41ad7"
server_key_iv: "ec19de5486c09ec2eeaf18b43864258f"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.429590 31282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:36.444730 31282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:36.449764 31282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:36.450105 31282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:36.451184 31282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:36.452781 31282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:36.452906 31282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:36.453081 31282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:36.453171 31282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:36.489547 31282 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:45079
I20260806 01:07:36.489601 31465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:45079 every 8 connection(s)
I20260806 01:07:36.490906 31282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:36.492154 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 31282
I20260806 01:07:36.492347 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:36.492923 22781 external_mini_cluster.cc:1428] Setting key 44f6b1e18b710e2cd2e44d63c19e30fd
Aug 06 01:07:36 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978456, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:36.514497 31469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:36.493865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:60165 (local address 127.22.63.126:40859)
0806 01:07:36.494331 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:36.494335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:36.495660 (+ 1325us) server_negotiation.cc:407] Connection header received
0806 01:07:36.498517 (+ 2857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:36.498522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:36.498593 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:36.498694 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:36.500893 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:36.501625 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:36.504145 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:36.504621 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:36.504728 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:36.508574 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:36.508579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:36.508583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:36.508617 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:36.511289 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:36.512110 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:36.512114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:36.512117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:36.512247 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:36.512711 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:36.512715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:36.512717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:36.512982 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:36.513151 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:36.513858 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 01:07:36.514100 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260806 01:07:36.515502 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:36.494575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40859 (local address 127.22.63.66:60165)
0806 01:07:36.495469 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:36.495528 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:36.498200 (+ 2672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:36.498972 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:36.498986 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:36.499819 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:36.500675 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:36.500706 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:36.501807 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:36.501815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:36.503973 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:36.503991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:36.504866 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:36.504875 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:36.505216 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:36.506046 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:36.506085 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:36.508325 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:36.511442 (+ 3117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:36.511450 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:36.511467 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:36.511974 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:36.512364 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:36.512369 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:36.512373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:36.512578 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:36.513126 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:36.513134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:36.513480 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 01:07:36.513753 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 01:07:36.514289 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":392,"threads_started":1}
I20260806 01:07:36.517074 31466 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40859
I20260806 01:07:36.517412 31466 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:36.518177 31466 heartbeater.cc:511] Master 127.22.63.126:40859 requested a full tablet report, sending...
I20260806 01:07:36.519778 30976 ts_manager.cc:194] Registered new tserver with Master: 8ca5c60b9c6e4d25937ba01794a83594 (127.22.63.66:45079)
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
I20260806 01:07:36.520673 30976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:60165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:07:36.554869 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:36.898911 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:36.899209 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:36.899367 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:36.906847 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:36.906980 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:36.907164 31476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:36.917181 31476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:36.918576 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:36.919986 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:36.933940 31485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:36.933909 31483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:36.934442 31482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:36.934834 31476 server_base.cc:1034] running on GCE node
I20260806 01:07:36.935510 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:36.936609 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:36.937934 31476 hybrid_clock.cc:648] HybridClock initialized: now 1785978456937863 us; error 61 us; skew 500 ppm
I20260806 01:07:36.938290 31476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:36 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978456, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:36.946141 31476 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:07:36.947881 31476 webserver.cc:460] Webserver started at http://127.22.63.67:43741/ using document root <none> and password file <none>
I20260806 01:07:36.948642 31476 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:36.948757 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:36.949059 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:36.952643 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "f7f301d9660244e29f95f532c8c5fec7"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "0493956462c03e2484afca0c1cc766e8"
server_key_iv: "5aeab02e88bc122f55f804610b5b8ce9"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.953644 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "f7f301d9660244e29f95f532c8c5fec7"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "0493956462c03e2484afca0c1cc766e8"
server_key_iv: "5aeab02e88bc122f55f804610b5b8ce9"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.960179 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260806 01:07:36.964027 31493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:36.965397 31476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:36.965585 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "f7f301d9660244e29f95f532c8c5fec7"
format_stamp: "Formatted at 2026-08-06 01:07:36 on dist-test-slave-7f01"
server_key: "0493956462c03e2484afca0c1cc766e8"
server_key_iv: "5aeab02e88bc122f55f804610b5b8ce9"
server_key_version: "encryptionkey@0"
I20260806 01:07:36.965802 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:36.984529 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:36.988873 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:36.989188 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:36.990191 31476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:36.990514 31274 heartbeater.cc:503] Master 127.22.63.126:40859 was elected leader, sending a full tablet report...
I20260806 01:07:36.991995 31476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:36.992074 31476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:36.992316 31476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:36.992446 31476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:37.027884 31476 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:45139
I20260806 01:07:37.027988 31644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:45139 every 8 connection(s)
I20260806 01:07:37.029274 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:37.035100 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 31476
I20260806 01:07:37.035290 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:37.035892 22781 external_mini_cluster.cc:1428] Setting key 2eb9bf4e48ea140eae85e02636ed4cc2
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978456, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:07:37.051303 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.031968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:54789 (local address 127.22.63.126:40859)
0806 01:07:37.032459 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.032463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.033363 (+ 900us) server_negotiation.cc:407] Connection header received
0806 01:07:37.035555 (+ 2192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.035560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.035655 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.035805 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.037893 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.039129 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.040873 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.041259 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.041355 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.045112 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.045118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.045123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.045163 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.047947 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.048680 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.048684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.048687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.048823 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.049227 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.049230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.049232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.049502 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.049627 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.050710 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.050922 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":245,"threads_started":1}
I20260806 01:07:37.052560 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.032418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40859 (local address 127.22.63.67:54789)
0806 01:07:37.033182 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:37.033224 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:37.035197 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:37.036080 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:37.036091 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:37.036817 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:37.037651 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:37.037680 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.039331 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.039337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:37.040736 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:37.040749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.041492 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.041498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:37.041703 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.042473 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:37.042497 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:37.044861 (+ 2364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:37.048107 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:37.048114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:37.048127 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:37.048553 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:37.048934 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:37.048937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:37.048939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:37.049117 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:37.049670 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:37.049680 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:37.050246 (+ 566us) client_negotiation.cc:769] Sending connection context
0806 01:07:37.050562 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 01:07:37.051146 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":301,"threads_started":1}
I20260806 01:07:37.053889 31646 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40859
I20260806 01:07:37.054193 31646 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:37.055032 31646 heartbeater.cc:511] Master 127.22.63.126:40859 requested a full tablet report, sending...
I20260806 01:07:37.056458 30976 ts_manager.cc:194] Registered new tserver with Master: f7f301d9660244e29f95f532c8c5fec7 (127.22.63.67:45139)
I20260806 01:07:37.057300 30976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:54789
I20260806 01:07:37.064611 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/test-admin.keytab.
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.097764 22781 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 01:07:37.103277 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:37.100638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.22.63.65:39543)
0806 01:07:37.101351 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.101357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.101386 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:07:37.101456 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.101464 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.101684 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.101816 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.102736 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45348: BlockingRecv error: recv got EOF from 127.0.0.1:45348 (error 108)
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":384,"threads_started":1}
W20260806 01:07:37.103826 22781 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.105598 22781 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.107028 22781 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:37.123212 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.108816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.22.63.65:39543)
0806 01:07:37.109035 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.109043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.109249 (+ 206us) server_negotiation.cc:407] Connection header received
0806 01:07:37.109480 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.109484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.109546 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.109654 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.110708 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.111683 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.112975 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.113554 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.113699 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.117337 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.117349 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.117363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.117404 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.120332 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.121048 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.121056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.121058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.121195 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.121595 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.121600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.121603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.121969 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.122291 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.122625 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.122908 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 01:07:37.129767 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:37.127824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.22.63.65:39543)
0806 01:07:37.128049 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.128054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.128320 (+ 266us) server_negotiation.cc:407] Connection header received
0806 01:07:37.128442 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.128446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.128506 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.128585 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.129472 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45366: BlockingRecv error: recv got EOF from 127.0.0.1:45366 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 01:07:37.129838 22781 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.131542 22781 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.133101 22781 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:37.147605 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.134576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.22.63.65:39543)
0806 01:07:37.134727 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.134731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.135157 (+ 426us) server_negotiation.cc:407] Connection header received
0806 01:07:37.135250 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.135254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.135325 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.135409 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.136484 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.137254 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.138457 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.138900 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.139018 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.142586 (+ 3568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.142591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.142594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.142636 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.145300 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.145927 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.145932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.145934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.146061 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.146401 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.146404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.146406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.146665 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.146885 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.147190 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.147372 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 01:07:37.152098 31658 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:37.150346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45396 (local address 127.22.63.65:39543)
0806 01:07:37.150492 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.150496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.150870 (+ 374us) server_negotiation.cc:407] Connection header received
0806 01:07:37.150989 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.150993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.151059 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.151168 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.151825 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45396: BlockingRecv error: recv got EOF from 127.0.0.1:45396 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 01:07:37.152304 22781 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785978402713339-22781-0/krb5kdc/krb5cc' of type FILE
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.154040 22781 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:37.155380 22781 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:07:37.171324 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.156791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.22.63.65:39543)
0806 01:07:37.157145 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.157152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.157458 (+ 306us) server_negotiation.cc:407] Connection header received
0806 01:07:37.157637 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.157641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.157697 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.157797 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.158733 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.159456 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.160765 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.161211 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.161364 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.165632 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.165638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.165643 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.165684 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.168222 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.168881 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.168887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.168890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.169052 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.169466 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.169471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.169474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.169816 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.170198 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.170467 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.170685 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260806 01:07:37.173066 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 31073
I20260806 01:07:37.185360 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.173792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.22.63.65:39543)
0806 01:07:37.173967 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.173971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.174362 (+ 391us) server_negotiation.cc:407] Connection header received
0806 01:07:37.174480 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.174483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.174549 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.174629 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.175597 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.176336 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.177621 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.178031 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.178138 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.180363 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.180368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.180372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.180401 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.182953 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.183589 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.183593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.183595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.183715 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.184082 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.184085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.184087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.184340 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.184535 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.184772 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.184931 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:07:37.185905 31261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:37.272059 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 31073
I20260806 01:07:37.298970 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 31282
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:07:37.315635 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.299957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.22.63.66:45079)
0806 01:07:37.300559 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.300566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.300592 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:07:37.300654 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.300661 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.300877 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.301004 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.301954 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.302946 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.304410 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.304970 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.305070 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.309000 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.309013 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.309031 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.309069 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.312678 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.313384 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.313393 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.313397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.313538 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.313952 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.313958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.313961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.314361 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.314665 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.314973 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.315231 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":308,"threads_started":1}
I20260806 01:07:37.317202 31460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:37.401489 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 31282
I20260806 01:07:37.427372 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 31476
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[30912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:07:37.443997 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:37.428473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.22.63.67:45139)
0806 01:07:37.429164 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:37.429171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:37.429203 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:07:37.429266 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:37.429273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:37.429485 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:37.429623 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:37.430946 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.432151 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.433351 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:37.433897 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:37.433995 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:37.438053 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:37.438065 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:37.438080 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:37.438114 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:37.441137 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.441865 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.441872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.441875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.442010 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:37.442357 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:37.442361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:37.442363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:37.442728 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:37.442999 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:37.443243 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 01:07:37.443450 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":376,"threads_started":1}
I20260806 01:07:37.445363 31640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:37.528697 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 31476
I20260806 01:07:37.552512 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 30932
I20260806 01:07:37.553781 31025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:37.659787 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 30932
2026-08-06T01:07:37Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4356 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[31690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[31690](info): set up 2 sockets
Aug 06 01:07:37 dist-test-slave-7f01 krb5kdc[31690](info): commencing operation
krb5kdc: starting...
W20260806 01:07:38.994546 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:39 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab.
Entry for principal oryx/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab.
I20260806 01:07:39.253516 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45699
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab
--principal=oryx/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:39.585683 31706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:39.586025 31706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:39.586189 31706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:39.593335 31706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:39.593494 31706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:39.593600 31706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:39.593703 31706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:39.593791 31706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:39.604278 31706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab
--principal=oryx/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:39.605669 31706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:39.607041 31706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:39.618777 31718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:39.618844 31713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:39.618880 31712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:39.620131 31706 server_base.cc:1034] running on GCE node
I20260806 01:07:39.620941 31706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:39.622349 31706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:39.623584 31706 hybrid_clock.cc:648] HybridClock initialized: now 1785978459623533 us; error 33 us; skew 500 ppm
I20260806 01:07:39.624014 31706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:39 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:39.627162 31706 init.cc:377] Logged in from keytab as oryx/127.22.63.126@KRBTEST.COM (short username oryx)
I20260806 01:07:39.628728 31706 webserver.cc:460] Webserver started at http://127.22.63.126:37081/ using document root <none> and password file <none>
I20260806 01:07:39.629443 31706 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:39.629590 31706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:39.629906 31706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:39.633001 31706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "3a3b57e3ad26495ebde108ea7f432867"
format_stamp: "Formatted at 2026-08-06 01:07:39 on dist-test-slave-7f01"
server_key: "b1c4bc22c2e62666e3e2d17c81033c89"
server_key_iv: "4e3e301360eefb5e5056ed3b04258e94"
server_key_version: "encryptionkey@0"
I20260806 01:07:39.633807 31706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "3a3b57e3ad26495ebde108ea7f432867"
format_stamp: "Formatted at 2026-08-06 01:07:39 on dist-test-slave-7f01"
server_key: "b1c4bc22c2e62666e3e2d17c81033c89"
server_key_iv: "4e3e301360eefb5e5056ed3b04258e94"
server_key_version: "encryptionkey@0"
I20260806 01:07:39.639286 31706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 01:07:39.642865 31729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:39.644245 31706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:39.644441 31706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "3a3b57e3ad26495ebde108ea7f432867"
format_stamp: "Formatted at 2026-08-06 01:07:39 on dist-test-slave-7f01"
server_key: "b1c4bc22c2e62666e3e2d17c81033c89"
server_key_iv: "4e3e301360eefb5e5056ed3b04258e94"
server_key_version: "encryptionkey@0"
I20260806 01:07:39.644649 31706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:39.665962 31706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:39.670364 31706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:39.670696 31706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:39.690259 31706 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:45699
I20260806 01:07:39.690332 31807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:45699 every 8 connection(s)
I20260806 01:07:39.691692 31706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:39.695559 31809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:39.697903 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 31706
I20260806 01:07:39.698132 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:39.698967 22781 external_mini_cluster.cc:1428] Setting key 9bee9608e8cc0c4cc9c8fb56ab2916a3
I20260806 01:07:39.709018 31809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Bootstrap starting.
I20260806 01:07:39.713017 31809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:39.714268 31809 log.cc:824] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:07:39 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:39.717991 31809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: No bootstrap required, opened a new log
I20260806 01:07:39.724579 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:39.701207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.22.63.126:45699)
0806 01:07:39.702129 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:39.702141 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:39.702184 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:07:39.703887 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:39.703911 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:39.704306 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:39.704636 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:39.706480 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:39.707889 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:39.709680 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:39.710441 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:39.710587 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:39.715851 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:39.715867 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:39.715890 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:39.715936 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:39.719983 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:39.720665 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:39.720673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:39.720681 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:39.720901 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:39.721318 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:39.721325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:39.721328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:39.722021 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:39.722368 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:39.722694 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 01:07:39.723100 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":426,"threads_started":1}
I20260806 01:07:39.727708 31809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } }
I20260806 01:07:39.728092 31809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:39.728166 31809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3b57e3ad26495ebde108ea7f432867, State: Initialized, Role: FOLLOWER
I20260806 01:07:39.728622 31809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } }
I20260806 01:07:39.728819 31809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:39.728919 31809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:39.729038 31809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:39.731166 31809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } }
I20260806 01:07:39.731594 31809 leader_election.cc:302] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a3b57e3ad26495ebde108ea7f432867; no voters:
I20260806 01:07:39.732266 31809 leader_election.cc:288] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:39.732421 31815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:39.733251 31815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 LEADER]: Becoming Leader. State: Replica: 3a3b57e3ad26495ebde108ea7f432867, State: Running, Role: LEADER
I20260806 01:07:39.733847 31815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } }
I20260806 01:07:39.734470 31809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:39.736627 31818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3b57e3ad26495ebde108ea7f432867. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } }
I20260806 01:07:39.736972 31818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:39.736735 31817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } }
I20260806 01:07:39.737107 31817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:39.738626 31827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:39.743675 31827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:39.752033 31827 catalog_manager.cc:1347] Generated new cluster ID: 86921016eb8a44149bd72d50d420a6a2
I20260806 01:07:39.752156 31827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:39.771703 31827 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:39.773653 31827 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:39.791924 31827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Generated new TSK 0
I20260806 01:07:39.793032 31827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
I20260806 01:07:39.863812 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:40.218613 31849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:40.218961 31849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:40.219125 31849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:40.226565 31849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:40.226718 31849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:40.226905 31849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:40.237205 31849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:40.238632 31849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:40.240170 31849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:40.254218 31860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:40.254266 31862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:40.254176 31857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:40.255110 31849 server_base.cc:1034] running on GCE node
I20260806 01:07:40.255860 31849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:40.256847 31849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:40.258127 31849 hybrid_clock.cc:648] HybridClock initialized: now 1785978460258024 us; error 99 us; skew 500 ppm
I20260806 01:07:40.258487 31849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:40 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978460, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:40.261756 31849 init.cc:377] Logged in from keytab as oryx/127.22.63.65@KRBTEST.COM (short username oryx)
I20260806 01:07:40.263218 31849 webserver.cc:460] Webserver started at http://127.22.63.65:40971/ using document root <none> and password file <none>
I20260806 01:07:40.264014 31849 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:40.264158 31849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:40.264467 31849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:40.267406 31849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "345b5bd3e2d745f399caa6fd83dc1670"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "bac7735471f6fb72c556f801c46060a1"
server_key_iv: "14bb1ca565dd341b9521b503367c4a06"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.268219 31849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "345b5bd3e2d745f399caa6fd83dc1670"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "bac7735471f6fb72c556f801c46060a1"
server_key_iv: "14bb1ca565dd341b9521b503367c4a06"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.273487 31849 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:40.277012 31869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.278326 31849 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:40.278513 31849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "345b5bd3e2d745f399caa6fd83dc1670"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "bac7735471f6fb72c556f801c46060a1"
server_key_iv: "14bb1ca565dd341b9521b503367c4a06"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.278723 31849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:40.313277 31849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:40.317575 31849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:40.317888 31849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:40.318884 31849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:40.320302 31849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:40.320415 31849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.320569 31849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:40.320642 31849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.355159 31849 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39387
I20260806 01:07:40.355218 32036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39387 every 8 connection(s)
I20260806 01:07:40.356611 31849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:40.365981 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 31849
I20260806 01:07:40.366216 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:40.366977 22781 external_mini_cluster.cc:1428] Setting key 90ed597e5bdcd158ef7cd22bee4a4a8b
Aug 06 01:07:40 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978460, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:40.378975 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:40.359269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:40849 (local address 127.22.63.126:45699)
0806 01:07:40.359899 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:40.359906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:40.360792 (+ 886us) server_negotiation.cc:407] Connection header received
0806 01:07:40.362611 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:40.362615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:40.362687 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:40.362784 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:40.364331 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.365103 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.366839 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.367475 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.367626 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:40.371373 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:40.371382 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:40.371389 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:40.371436 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:40.375289 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:40.376172 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:40.376180 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:40.376186 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:40.376360 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:40.376793 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:40.376800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:40.376806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:40.377220 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:40.377398 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:40.378222 (+ 824us) server_negotiation.cc:299] Negotiation successful
0806 01:07:40.378497 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":243,"threads_started":1}
I20260806 01:07:40.379685 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:40.359897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:40849)
0806 01:07:40.360624 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:40.360662 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:40.362359 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:40.363027 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:40.363035 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:40.363543 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:40.364153 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:40.364170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.365270 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.365275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:40.366666 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:40.366679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.367808 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.367818 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:40.368127 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:40.368854 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:40.368883 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:40.371131 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:40.375484 (+ 4353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:40.375492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:40.375507 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:40.376033 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:40.376479 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:40.376482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:40.376484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:40.376673 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:40.377383 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:40.377397 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:40.377779 (+ 382us) client_negotiation.cc:769] Sending connection context
0806 01:07:40.378051 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 01:07:40.378488 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":280,"threads_started":1}
I20260806 01:07:40.381472 32038 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:40.381805 32038 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:40.382783 32038 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:40.385095 31752 ts_manager.cc:194] Registered new tserver with Master: 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:40.386587 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:40849
WARNING: no policy specified for oryx/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
I20260806 01:07:40.427865 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:40.790385 32050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:40.790669 32050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:40.790828 32050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:40.797855 32050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:40.797956 32050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:40.798122 32050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:40.807479 32050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:40.808786 32050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:40.810104 32050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:40.823618 32058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:40.823552 32063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:40.823613 32059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:40.824374 32050 server_base.cc:1034] running on GCE node
I20260806 01:07:40.824927 32050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:40.825809 32050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:40.827014 32050 hybrid_clock.cc:648] HybridClock initialized: now 1785978460826975 us; error 31 us; skew 500 ppm
I20260806 01:07:40.827337 32050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:40 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978460, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:40.830583 32050 init.cc:377] Logged in from keytab as oryx/127.22.63.66@KRBTEST.COM (short username oryx)
I20260806 01:07:40.832058 32050 webserver.cc:460] Webserver started at http://127.22.63.66:38353/ using document root <none> and password file <none>
I20260806 01:07:40.832739 32050 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:40.832873 32050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:40.833191 32050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:40.836256 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "84087abafdfd4311b171b1e151889812"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "3c23d7164b97db0a242f255bdf4db048"
server_key_iv: "98253caef1af1e8cf57399f5ceb445d2"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.837042 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "84087abafdfd4311b171b1e151889812"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "3c23d7164b97db0a242f255bdf4db048"
server_key_iv: "98253caef1af1e8cf57399f5ceb445d2"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.842363 32050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:40.846001 32071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.847472 32050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:40.847656 32050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "84087abafdfd4311b171b1e151889812"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "3c23d7164b97db0a242f255bdf4db048"
server_key_iv: "98253caef1af1e8cf57399f5ceb445d2"
server_key_version: "encryptionkey@0"
I20260806 01:07:40.847940 32050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:40.893213 32050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:40.897538 32050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:40.897846 32050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:40.898860 32050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:40.900309 32050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:40.900391 32050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.900552 32050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:40.900605 32050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:40.935977 32050 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:46621
I20260806 01:07:40.936033 32232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:46621 every 8 connection(s)
I20260806 01:07:40.937426 32050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:40.940697 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 32050
I20260806 01:07:40.940902 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:40.941450 22781 external_mini_cluster.cc:1428] Setting key 1609fd3c61bdf1200e050f71f5679a62
Aug 06 01:07:40 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978460, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:40.960393 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:40.940277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59693 (local address 127.22.63.126:45699)
0806 01:07:40.940838 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:40.940847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:40.942067 (+ 1220us) server_negotiation.cc:407] Connection header received
0806 01:07:40.944130 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:40.944135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:40.944222 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:40.944347 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:40.945854 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.947046 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.948775 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.949573 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.949722 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:40.953554 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:40.953562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:40.953570 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:40.953625 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:40.956593 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:40.957615 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:40.957620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:40.957622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:40.957770 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:40.958375 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:40.958382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:40.958387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:40.958726 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:40.958905 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:40.959901 (+ 996us) server_negotiation.cc:299] Negotiation successful
0806 01:07:40.960091 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":257,"threads_started":1}
I20260806 01:07:40.961927 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:40.940984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:59693)
0806 01:07:40.941858 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:40.941914 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:40.943857 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:40.944563 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:40.944571 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:40.945076 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:40.945665 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:40.945682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.947204 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.947208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:40.948632 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:40.948644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:40.949789 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:40.949794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:40.950010 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:40.950701 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:40.950725 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:40.953296 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:40.956743 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:40.956755 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:40.956775 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:40.957457 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:40.957893 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:40.957900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:40.957906 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:40.958201 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:40.958860 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:40.958871 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:40.959377 (+ 506us) client_negotiation.cc:769] Sending connection context
0806 01:07:40.959702 (+ 325us) client_negotiation.cc:240] Negotiation successful
0806 01:07:40.960386 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":316,"threads_started":1}
I20260806 01:07:40.963634 32233 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:40.964018 32233 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:40.964789 32233 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:40.966305 31752 ts_manager.cc:194] Registered new tserver with Master: 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:40.967245 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:59693
WARNING: no policy specified for oryx/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
I20260806 01:07:41.007158 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:41.345284 32243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:41.345573 32243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:41.345671 32243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:41.353178 32243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:41.353296 32243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:41.353416 32243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:41.363306 32243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:41.364499 32243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:41.365736 32243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:41.378448 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:41.378666 32251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:41.378695 32254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:41.379612 32243 server_base.cc:1034] running on GCE node
I20260806 01:07:41.380213 32243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:41.381129 32243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:41.382500 32243 hybrid_clock.cc:648] HybridClock initialized: now 1785978461382444 us; error 48 us; skew 500 ppm
I20260806 01:07:41.382889 32243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:41 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978461, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:41.386252 32243 init.cc:377] Logged in from keytab as oryx/127.22.63.67@KRBTEST.COM (short username oryx)
I20260806 01:07:41.387990 32243 webserver.cc:460] Webserver started at http://127.22.63.67:46125/ using document root <none> and password file <none>
I20260806 01:07:41.388728 32243 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:41.388883 32243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:41.389207 32243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:41.390241 32038 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
I20260806 01:07:41.392613 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "3fd347473895fd7a8c40ed44ffcf05e4"
server_key_iv: "e37ccc626a10ca32c4b89005ba5724a0"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.393436 32243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "3fd347473895fd7a8c40ed44ffcf05e4"
server_key_iv: "e37ccc626a10ca32c4b89005ba5724a0"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.398779 32243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 01:07:41.402602 32261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:41.404044 32243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:07:41.404224 32243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "3fd347473895fd7a8c40ed44ffcf05e4"
server_key_iv: "e37ccc626a10ca32c4b89005ba5724a0"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.404443 32243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:41.428336 32243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:41.432940 32243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:41.433233 32243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:41.434262 32243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:41.435684 32243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:41.435828 32243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:07:41.435988 32243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:41.436060 32243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:41.469985 32243 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:39683
I20260806 01:07:41.470075 32416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:39683 every 8 connection(s)
I20260806 01:07:41.471329 32243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:41.473219 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 32243
I20260806 01:07:41.473412 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:41.473927 22781 external_mini_cluster.cc:1428] Setting key 15f96d6d12bfd750a66ac76ed5e52fce
Aug 06 01:07:41 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978461, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:41.491398 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:41.473983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:58937 (local address 127.22.63.126:45699)
0806 01:07:41.474501 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:41.474507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:41.475607 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 01:07:41.477420 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:41.477425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:41.477507 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:41.477607 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:41.478998 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:41.479887 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:41.481398 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:41.481799 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:41.481957 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:41.485305 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:41.485310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:41.485313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:41.485345 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:41.488447 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:41.489128 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:41.489135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:41.489139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:41.489267 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:41.489683 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:41.489689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:41.489692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:41.489957 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:41.490130 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:41.490786 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 01:07:41.491010 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":222,"threads_started":1}
I20260806 01:07:41.492417 32422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:41.474590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:58937)
0806 01:07:41.475443 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:41.475496 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:41.477170 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:41.477795 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:41.477803 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:41.478276 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:41.478831 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:41.478847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:41.480062 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:41.480066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:41.481273 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:41.481285 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:41.482015 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:41.482020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:41.482205 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:41.482870 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:41.482892 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:41.485125 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:41.488589 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:41.488596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:41.488608 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:41.489008 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:41.489379 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:41.489383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:41.489387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:41.489573 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:41.490094 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:41.490101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:41.490419 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 01:07:41.490693 (+ 274us) client_negotiation.cc:240] Negotiation successful
0806 01:07:41.491212 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":350,"threads_started":1}
I20260806 01:07:41.493727 32417 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:41.494055 32417 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:41.494788 32417 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:41.496202 31752 ts_manager.cc:194] Registered new tserver with Master: c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:41.497021 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:58937
I20260806 01:07:41.501406 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.125.keytab.
Entry for principal oryx/127.22.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.125.keytab.
I20260806 01:07:41.558917 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:35243
--webserver_interface=127.22.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.125.keytab
--principal=oryx/127.22.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45699,127.22.63.125:35243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:41.905164 32425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:41.905452 32425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:41.905542 32425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:41.912940 32425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:41.913048 32425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:41.913095 32425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:41.913153 32425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:41.913197 32425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:41.922998 32425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45699,127.22.63.125:35243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.125.keytab
--principal=oryx/127.22.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:35243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
W20260806 01:07:41.923336 32425 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.63.126:45699,127.22.63.125:35243'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 01:07:41.924445 32425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:41.925788 32425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:41.937808 32433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:41.938651 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:41.939052 32432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:41.939193 32425 server_base.cc:1034] running on GCE node
I20260806 01:07:41.940407 32425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:41.942229 32425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:41.943492 32425 hybrid_clock.cc:648] HybridClock initialized: now 1785978461943409 us; error 57 us; skew 500 ppm
I20260806 01:07:41.943912 32425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:41 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978461, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:41.950888 32425 init.cc:377] Logged in from keytab as oryx/127.22.63.125@KRBTEST.COM (short username oryx)
I20260806 01:07:41.952486 32425 webserver.cc:460] Webserver started at http://127.22.63.125:39327/ using document root <none> and password file <none>
I20260806 01:07:41.953258 32425 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:41.953356 32425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:41.953694 32425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:41.956722 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/instance:
uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "6c2e0e1b8859903bad0a5a97b71781f0"
server_key_iv: "a731c316a10cc824dc6b59a76b6af08b"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.957525 32425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal/instance:
uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "6c2e0e1b8859903bad0a5a97b71781f0"
server_key_iv: "a731c316a10cc824dc6b59a76b6af08b"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.962738 32425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 01:07:41.966198 32442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:41.967567 32425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:07:41.967785 32425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "6c2e0e1b8859903bad0a5a97b71781f0"
server_key_iv: "a731c316a10cc824dc6b59a76b6af08b"
server_key_version: "encryptionkey@0"
I20260806 01:07:41.968019 32425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:41.970000 32233 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
I20260806 01:07:41.985785 32425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:41.989952 32425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:41.990222 32425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:42.009493 32425 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.125:35243
I20260806 01:07:42.009548 32516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.125:35243 every 8 connection(s)
I20260806 01:07:42.010865 32425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
I20260806 01:07:42.014421 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 32425
I20260806 01:07:42.014616 32517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:42.023317 32517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978461, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.125@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:42.038508 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.016567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52204 (local address 127.22.63.125:35243)
0806 01:07:42.017360 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.017374 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.017416 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:07:42.019354 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.019376 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.019810 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.020211 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.021686 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.022933 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.024228 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.025201 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.025396 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.030124 (+ 4728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.030132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.030141 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.030180 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.033927 (+ 3747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.034671 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.034680 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.034689 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.034921 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.035377 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.035380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.035382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.036125 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.036353 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.036957 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.037384 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":373,"threads_started":1}
I20260806 01:07:42.042919 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.026200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.22.63.126:45699)
0806 01:07:42.026749 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.026758 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.027330 (+ 572us) server_negotiation.cc:407] Connection header received
0806 01:07:42.027521 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.027528 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.027623 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.027773 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.028837 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.029634 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.031201 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.031626 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.031799 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.035910 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.035915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.035918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.035951 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.040002 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.041027 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.041031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.041034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.041185 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.041667 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.041671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.041673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.041986 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.042115 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.042505 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.042671 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":288,"threads_started":1}
I20260806 01:07:42.042948 32524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.026230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33052)
0806 01:07:42.027166 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.027216 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.027384 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.027864 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.027869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.028423 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.028671 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.028680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.029774 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.029781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.031051 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.031068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.031860 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.031871 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.032103 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.032727 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.032757 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.035696 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.040268 (+ 4572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.040279 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.040285 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.040885 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.041344 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.041348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.041350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.041507 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.042104 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.042109 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.042255 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.042421 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.042666 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":568,"threads_started":1}
I20260806 01:07:42.044613 32517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978461, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.125@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
I20260806 01:07:42.057107 32524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.045579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52210)
0806 01:07:42.045784 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.045805 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.045956 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.046231 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.046236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.046603 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.046724 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.046732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.047698 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.047704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.048602 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.048614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.049318 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.049323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.049545 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.050101 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.050118 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.052304 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.054983 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.054987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.054990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.055395 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.055704 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.055707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.055709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.055901 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.056384 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.056388 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.056520 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.056634 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.056867 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":5}
I20260806 01:07:42.057498 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.045562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.22.63.125:35243)
0806 01:07:42.045784 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.045790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.045894 (+ 104us) server_negotiation.cc:407] Connection header received
0806 01:07:42.046070 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.046074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.046123 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.046213 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.046860 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.047562 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.048737 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.049176 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.049307 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.052455 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.052459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.052463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.052490 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.054858 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.055493 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.055497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.055499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.055611 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.055996 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.055999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.056001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.056255 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.056373 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.056904 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.057099 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":85}
I20260806 01:07:42.062434 32517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Bootstrap starting.
I20260806 01:07:42.065671 32517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:42.066725 32517 log.cc:824] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:42.069626 32517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: No bootstrap required, opened a new log
I20260806 01:07:42.076968 32517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } }
I20260806 01:07:42.077325 32517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:42.077433 32517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5086e71fbb3845c0a95b6ab29d26d1c8, State: Initialized, Role: FOLLOWER
I20260806 01:07:42.077998 32517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } }
I20260806 01:07:42.078838 32527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } }
I20260806 01:07:42.079147 32527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:42.079919 32517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:07:42.088346 32539 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:07:42.088491 32539 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:07:42.091354 32425 master_runner.cc:428] Detected that this master 5086e71fbb3845c0a95b6ab29d26d1c8 is joining an existing cluster
I20260806 01:07:42.091471 32425 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.63.125:35243
WARNING: no policy specified for oryx/127.22.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.124@KRBTEST.COM" created.
I20260806 01:07:42.135648 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.095862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.22.63.126:45699)
0806 01:07:42.096072 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.096080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.107239 (+ 11159us) server_negotiation.cc:407] Connection header received
0806 01:07:42.107705 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.107714 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.107876 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.108019 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.120097 (+ 12078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.121252 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.123482 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.124222 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.124364 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.128453 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.128460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.128465 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.128510 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.132137 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133152 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.133158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.133161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.133329 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133822 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.133827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.133831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.134198 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.134339 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.135022 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.135267 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 01:07:42.135653 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.095343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52218 (local address 127.22.63.125:35243)
0806 01:07:42.095561 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.095569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.118839 (+ 23270us) server_negotiation.cc:407] Connection header received
0806 01:07:42.119099 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.119111 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.119201 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.119285 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.120935 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.122219 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.124107 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.124954 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.125111 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.127581 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.127594 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.127602 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.127647 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.132032 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133057 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.133065 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.133071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.133215 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133641 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.133647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.133653 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.134126 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.134277 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.134900 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.135152 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":61}
I20260806 01:07:42.137076 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.095394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33054)
0806 01:07:42.107084 (+ 11690us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.107112 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.107388 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.108058 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.108063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.119298 (+ 11235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.119842 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.119862 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.121457 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.121462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.123306 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.123321 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.124452 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.124459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.124666 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.125458 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.125487 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.128207 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.132311 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.132316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.132320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.132993 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.133498 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.133506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.133684 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.134447 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.134454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.134725 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.136181 (+ 1456us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.136545 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11512,"mutex_wait_us":57,"spinlock_wait_cycles":25856,"thread_start_us":22853,"threads_started":1}
I20260806 01:07:42.137672 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.095886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52218)
0806 01:07:42.118678 (+ 22792us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.118704 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.118909 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.119583 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.119592 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.120363 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.120744 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.120761 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.122452 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.122459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.123934 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.123949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.125380 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.125386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.125567 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.126212 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.126250 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.127324 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.132196 (+ 4872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.132200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.132204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.132934 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.133349 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.133355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.133361 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.133527 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.134354 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.134361 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.134535 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.136262 (+ 1727us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.136784 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22597,"mutex_wait_us":154,"spinlock_wait_cycles":30592,"thread_start_us":10579,"threads_started":1}
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.126@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
I20260806 01:07:42.164045 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.144943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.22.63.125:35243)
0806 01:07:42.145136 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.145144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.145933 (+ 789us) server_negotiation.cc:407] Connection header received
0806 01:07:42.146315 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.146322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.146449 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.146567 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.148145 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.149370 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.151290 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.152130 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.152275 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.156855 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.156862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.156867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.156919 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.160837 (+ 3918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.161676 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.161683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.161686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.161847 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.162315 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.162319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.162321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.162680 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.162804 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.163351 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.163639 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.124.keytab.
Entry for principal oryx/127.22.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.124.keytab.
I20260806 01:07:42.164453 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.145061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52230)
0806 01:07:42.145793 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.145845 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.146074 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.146630 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.146638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.147583 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.147917 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.147931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.149514 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.149520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.151125 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.151141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.152307 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.152316 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.152600 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.153404 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.153449 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.156596 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.161051 (+ 4455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.161063 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.161069 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.161528 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.162001 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.162006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.162010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.162190 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.162821 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.162830 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.163049 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.163266 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.163635 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":302,"threads_started":1}
I20260806 01:07:42.165704 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:38357
--webserver_interface=127.22.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.124.keytab
--principal=oryx/127.22.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:07:42.167374 31752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.63.125:35243: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
member_type: NON_VOTER
last_known_addr {
host: "127.22.63.125"
port: 35243
}
attrs {
promote: true
}
}
dest_uuid: "3a3b57e3ad26495ebde108ea7f432867"
cas_config_opid_index: -1
I20260806 01:07:42.168959 31752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } }
I20260806 01:07:42.180363 32483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:42.183897 32483 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 01:07:42.179289 32556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.184998 32555 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:42.199990 32556 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.63.125:35243
I20260806 01:07:42.200793 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.201009 32425 master.cc:537] Master@127.22.63.125:35243 shutting down...
I20260806 01:07:42.201119 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.224996 32483 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.253283 32527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3b57e3ad26495ebde108ea7f432867. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.253583 32527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:42.252756 32572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.254010 32572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:42.253438 32565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.254663 32565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:42.257736 32425 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 01:07:42.258183 32425 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 01:07:42.258358 32425 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: stopping tablet replica
I20260806 01:07:42.266289 32425 master.cc:552] Master@127.22.63.125:35243 shutdown complete.
I20260806 01:07:42.266433 32425 master_runner.cc:305] Clearing existing system tablet
I20260806 01:07:42.266887 32425 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:42.267012 32425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:42.269352 32425 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 01:07:42.270864 32575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:42.271170 32425 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:42.271473 32425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 01:07:42.271564 32425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "6c2e0e1b8859903bad0a5a97b71781f0"
server_key_iv: "a731c316a10cc824dc6b59a76b6af08b"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.274844 32425 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:42.279663 32425 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:07:42.279899 32425 log.cc:1196] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 01:07:42.280388 32425 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Deleting consensus metadata
I20260806 01:07:42.280658 32425 master_runner.cc:315] Copying system tablet from 127.22.63.126:45699
I20260806 01:07:42.283776 32425 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: tablet copy: Beginning tablet copy session from remote peer at address 127.22.63.126:45699
I20260806 01:07:42.297423 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.284717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.22.63.126:45699)
0806 01:07:42.285006 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.285016 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.285561 (+ 545us) server_negotiation.cc:407] Connection header received
0806 01:07:42.285801 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.285807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.285889 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.286004 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.286869 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.287622 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.289057 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.289504 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.289644 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.291727 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.291732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.291773 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.291815 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.294538 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.295190 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.295196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.295200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.295338 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.295778 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.295784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.295787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.296191 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.296359 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.296759 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.296986 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:07:42.297485 32582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.284708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33056)
0806 01:07:42.285439 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.285461 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.285633 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.286031 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.286036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.286474 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.286694 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.286704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.287877 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.287887 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.288907 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.288920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.289662 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.289668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.289900 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.290580 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.290599 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.291507 (+ 908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.294685 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.294691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.294695 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.295070 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.295453 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.295458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.295462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.295624 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.296321 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.296325 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.296497 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.296664 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.296995 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":456,"threads_started":1}
I20260806 01:07:42.298758 31789 tablet_copy_service.cc:140] P 3a3b57e3ad26495ebde108ea7f432867: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5086e71fbb3845c0a95b6ab29d26d1c8 ({username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:33056)
I20260806 01:07:42.299237 31789 tablet_copy_service.cc:161] P 3a3b57e3ad26495ebde108ea7f432867: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5086e71fbb3845c0a95b6ab29d26d1c8 at {username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:33056: session id = 5086e71fbb3845c0a95b6ab29d26d1c8-00000000000000000000000000000000
I20260806 01:07:42.304112 31789 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:07:42.306862 32425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:07:42.314109 32425 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: tablet copy: Starting download of 0 data blocks...
I20260806 01:07:42.314508 32425 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: tablet copy: Starting download of 1 WAL segments...
I20260806 01:07:42.318115 32425 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:07:42.323045 31789 tablet_copy_service.cc:342] P 3a3b57e3ad26495ebde108ea7f432867: Request end of tablet copy session 5086e71fbb3845c0a95b6ab29d26d1c8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:33056
I20260806 01:07:42.323343 31789 tablet_copy_service.cc:433] P 3a3b57e3ad26495ebde108ea7f432867: ending tablet copy session 5086e71fbb3845c0a95b6ab29d26d1c8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5086e71fbb3845c0a95b6ab29d26d1c8
I20260806 01:07:42.336333 32425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:42.339174 32583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:42.339506 32584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:42.347250 32588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:42.353138 32425 server_base.cc:1034] running on GCE node
I20260806 01:07:42.353698 32425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:42.354589 32425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:42.358193 32425 hybrid_clock.cc:648] HybridClock initialized: now 1785978462358115 us; error 43 us; skew 500 ppm
I20260806 01:07:42.358469 32425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978462, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:42.360788 32425 init.cc:377] Logged in from keytab as oryx/127.22.63.125@KRBTEST.COM (short username oryx)
I20260806 01:07:42.362287 32425 webserver.cc:460] Webserver started at http://127.22.63.125:35821/ using document root <none> and password file <none>
I20260806 01:07:42.362691 32425 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:42.362774 32425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:42.366005 32425 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:07:42.368168 32597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:42.368609 32425 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 01:07:42.368737 32425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "6c2e0e1b8859903bad0a5a97b71781f0"
server_key_iv: "a731c316a10cc824dc6b59a76b6af08b"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.368911 32425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:42.387925 32425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:42.392100 32425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:42.430290 32425 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.125:35243
I20260806 01:07:42.430482 32670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.125:35243 every 8 connection(s)
I20260806 01:07:42.431257 32425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
I20260806 01:07:42.436480 32671 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:07:42.441311 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Bootstrap starting.
I20260806 01:07:42.460237 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:07:42.460868 32671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Bootstrap complete.
I20260806 01:07:42.462222 32671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } }
I20260806 01:07:42.462615 32671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5086e71fbb3845c0a95b6ab29d26d1c8, State: Initialized, Role: LEARNER
I20260806 01:07:42.462946 32671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } }
I20260806 01:07:42.464270 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.464535 32671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:42.464712 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:42.483120 32693 catalog_manager.cc:1259] Loaded cluster ID: 86921016eb8a44149bd72d50d420a6a2
I20260806 01:07:42.483246 32693 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: loading cluster ID for follower catalog manager: success
I20260806 01:07:42.486876 32693 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: acquiring CA information for follower catalog manager: success
I20260806 01:07:42.489398 32693 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:07:42.499562 32417 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
I20260806 01:07:42.599215 32556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.599555 32556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.608486 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.599531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52232)
0806 01:07:42.599917 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.599942 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.600100 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.601263 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.601271 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.601316 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:42.601537 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.601549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.603701 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.603708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.605808 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.605826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.607527 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.607534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.607793 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.607863 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.607964 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.608114 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260806 01:07:42.608968 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.600029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52232 (local address 127.22.63.125:35243)
0806 01:07:42.600756 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.600766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.600798 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:07:42.600892 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.600899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.601014 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.601169 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:42.601728 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.603517 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.605977 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.607343 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.607469 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.607809 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.608018 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.608160 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":344,"threads_started":1}
I20260806 01:07:42.609997 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3b57e3ad26495ebde108ea7f432867. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.610257 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:42.611127 32559 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: attempting to promote NON_VOTER 5086e71fbb3845c0a95b6ab29d26d1c8 to VOTER
I20260806 01:07:42.612106 32559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } }
W20260806 01:07:42.612784 32557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:42.613169 32557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:42.613322 32557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 01:07:42.614241 32635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 LEARNER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 01:07:42.614781 32556 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 01:07:42.617887 32559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } }
I20260806 01:07:42.622007 32556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } }
I20260806 01:07:42.622223 32556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.622469 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } }
I20260806 01:07:42.622653 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.622380 32635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } }
W20260806 01:07:42.623981 32557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:42.624106 32557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:42.624204 32557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:42.624289 32557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:42.624354 32557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:42.625746 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } }
I20260806 01:07:42.626004 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:42.634677 32557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.124.keytab
--principal=oryx/127.22.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:38357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:42.636152 32557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:42.637477 32557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:42.649928 32702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:42.649914 32703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:42.650095 32705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:42.650908 32557 server_base.cc:1034] running on GCE node
I20260806 01:07:42.651714 32557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:42.653491 32557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:42.654734 32557 hybrid_clock.cc:648] HybridClock initialized: now 1785978462654693 us; error 36 us; skew 500 ppm
I20260806 01:07:42.655135 32557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978462, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:42.658413 32557 init.cc:377] Logged in from keytab as oryx/127.22.63.124@KRBTEST.COM (short username oryx)
I20260806 01:07:42.659988 32557 webserver.cc:460] Webserver started at http://127.22.63.124:45633/ using document root <none> and password file <none>
I20260806 01:07:42.660681 32557 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:42.660828 32557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:42.661127 32557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:42.664240 32557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/instance:
uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
format_stamp: "Formatted at 2026-08-06 01:07:42 on dist-test-slave-7f01"
server_key: "47ec1f76d6bd8837e72bd15d30fd1838"
server_key_iv: "cfa381879903b3dd9f0fb2b66240faac"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.665035 32557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal/instance:
uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
format_stamp: "Formatted at 2026-08-06 01:07:42 on dist-test-slave-7f01"
server_key: "47ec1f76d6bd8837e72bd15d30fd1838"
server_key_iv: "cfa381879903b3dd9f0fb2b66240faac"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.670593 32557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:42.674152 32717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:42.675465 32557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:42.675649 32557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
format_stamp: "Formatted at 2026-08-06 01:07:42 on dist-test-slave-7f01"
server_key: "47ec1f76d6bd8837e72bd15d30fd1838"
server_key_iv: "cfa381879903b3dd9f0fb2b66240faac"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.675921 32557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:42.710152 32557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:42.714789 32557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:42.715090 32557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:42.734086 32557 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.124:38357
I20260806 01:07:42.734167 321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.124:38357 every 8 connection(s)
I20260806 01:07:42.735530 32557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
I20260806 01:07:42.739249 323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:42.741499 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 32557
I20260806 01:07:42.749606 323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978462, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.124@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:42.766786 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.743877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.22.63.124:38357)
0806 01:07:42.744716 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.744737 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.744803 (+ 66us) server_negotiation.cc:407] Connection header received
0806 01:07:42.746579 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.746614 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.747144 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.747596 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.749132 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.751168 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.752664 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.753666 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.753895 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.758845 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.758853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.758867 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.758903 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.762637 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.763305 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.763319 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.763327 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.763474 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.763921 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.763935 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.763937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.764500 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.764833 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.765286 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.765698 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":337,"threads_started":1}
I20260806 01:07:42.769479 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.752979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.22.63.126:45699)
0806 01:07:42.753198 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.753205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.753921 (+ 716us) server_negotiation.cc:407] Connection header received
0806 01:07:42.754194 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.754199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.754274 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.754389 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.755941 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.756688 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.758254 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.758895 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.758988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.763105 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.763110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.763113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.763145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.766035 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.766730 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.766738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.766742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.766939 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.768049 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.768052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.768054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.768306 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.768429 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.768963 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.769129 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:07:42.769816 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.752880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33060)
0806 01:07:42.753783 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.753832 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.754052 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.754456 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.754461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.755376 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.755766 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.755802 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.756830 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.756836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.758108 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.758120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.759096 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.759102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.759302 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.760085 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.760120 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.762930 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.766159 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.766171 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.766177 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.766614 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.767078 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.767083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.767086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.767266 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.768459 (+ 1193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.768466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.768649 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.768885 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.769200 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":495,"threads_started":1}
I20260806 01:07:42.771230 323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978462, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.124@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
I20260806 01:07:42.784528 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.772288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52248)
0806 01:07:42.772525 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.772566 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.772653 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.772970 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.772973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.773316 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.773454 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.773462 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.774439 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.774443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.775483 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.775499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.776539 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.776544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.776720 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.777290 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.777307 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.779364 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.782354 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.782359 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.782363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.782803 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.783153 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.783156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.783158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.783284 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.783868 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.783872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.783987 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.784097 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.784298 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 01:07:42.784611 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.772402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.22.63.125:35243)
0806 01:07:42.772604 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.772609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.772626 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:07:42.772747 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.772751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.772810 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.772907 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.773591 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.774315 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.775619 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.776388 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.776527 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.779524 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.779528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.779532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.779565 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.782209 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.782911 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.782915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.782917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.783056 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.783377 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.783380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.783381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.783696 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.783899 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.784191 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.784366 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:07:42.785519 323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978462, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.124@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
I20260806 01:07:42.797549 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.786132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.0.0.1:60576)
0806 01:07:42.786283 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.786304 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.786376 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.786600 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.786603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.786923 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.787063 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.787071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.788029 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.788032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.788836 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.788845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.789549 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.789553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.789715 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.790240 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.790256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.792176 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.795418 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.795422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.795425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.795873 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.796185 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.796188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.796191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.796338 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.796794 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.796797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.796939 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.797053 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.797250 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 01:07:42.797664 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.786202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.22.63.124:38357)
0806 01:07:42.786342 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.786346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.786362 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:07:42.786445 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.786448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.786496 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.786570 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.787175 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.787916 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.788984 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.789401 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.789508 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.792358 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.792363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.792366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.792395 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.795288 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.795976 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.795980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.795982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.796092 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.796428 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.796431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.796433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.796694 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.796802 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.797131 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.797381 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":47}
I20260806 01:07:42.802371 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Bootstrap starting.
I20260806 01:07:42.805593 323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:42.806641 323 log.cc:824] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:42.809537 323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: No bootstrap required, opened a new log
I20260806 01:07:42.816606 323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } }
I20260806 01:07:42.816931 323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:42.817045 323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d777326282e4cd99dcd7fd6ae52cfae, State: Initialized, Role: FOLLOWER
I20260806 01:07:42.817606 323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } }
I20260806 01:07:42.818467 330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } } }
I20260806 01:07:42.818796 330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:42.819459 323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:07:42.828593 347 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:07:42.828737 347 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:07:42.833508 32557 master_runner.cc:428] Detected that this master 2d777326282e4cd99dcd7fd6ae52cfae is joining an existing cluster
I20260806 01:07:42.833670 32557 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.63.124:38357
I20260806 01:07:42.857753 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.838167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.22.63.126:45699)
0806 01:07:42.838360 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.838366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.838802 (+ 436us) server_negotiation.cc:407] Connection header received
0806 01:07:42.839025 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.839031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.839119 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.839229 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.841009 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.842233 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.844538 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.845204 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.845306 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.850673 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.850680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.850685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.850730 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.854359 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.855201 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.855207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.855211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.855368 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.855907 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.855912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.855916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.856310 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.856449 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.857147 (+ 698us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.857378 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:07:42.858348 32694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.837713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.22.63.125:35243)
0806 01:07:42.838027 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.838036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.839634 (+ 1598us) server_negotiation.cc:407] Connection header received
0806 01:07:42.840006 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.840015 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.840117 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.840208 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.842297 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.843584 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.846322 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.847239 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.847374 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.851009 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.851018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.851025 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.851076 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.854979 (+ 3903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.855798 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.855804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.855808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.855978 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.856492 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.856497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.856500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.856864 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.857024 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.857633 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.857915 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169}
I20260806 01:07:42.859516 356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.837784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33072)
0806 01:07:42.838663 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.838688 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.838874 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.839417 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.839422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.840382 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.840817 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.840832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.842462 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.842467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.844347 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.844364 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.845645 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.845653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.845969 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.847955 (+ 1986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.847987 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.850438 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.854531 (+ 4093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.854537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.854543 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.855056 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.855521 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.855526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.855529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.855728 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.856536 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.856541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.856821 (+ 280us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.858282 (+ 1461us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.858995 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":34,"thread_start_us":579,"threads_started":1}
I20260806 01:07:42.859789 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.837795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52256)
0806 01:07:42.839526 (+ 1731us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.839547 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.839845 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.840498 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.840506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.841599 (+ 1093us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.842066 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.842082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.843814 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.843819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.846095 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.846116 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.847931 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.847942 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.848128 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.849062 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.849086 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.850757 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.855139 (+ 4382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.855144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.855148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.855627 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.856134 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.856141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.856145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.856362 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.857116 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.857122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.857334 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.858611 (+ 1277us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.859202 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"mutex_wait_us":279,"thread_start_us":1367,"threads_started":1}
I20260806 01:07:42.866933 327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.839649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.22.63.124:38357)
0806 01:07:42.839913 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.839919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.850162 (+ 10243us) server_negotiation.cc:407] Connection header received
0806 01:07:42.850373 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.850378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.850463 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.850528 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.852477 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.854130 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.856158 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.856887 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.856976 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.859594 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.859600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.859605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.859637 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.863988 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.864797 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.864803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.864806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.864966 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.865378 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.865382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.865385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.865760 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.865950 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.866353 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.866570 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 01:07:42.868152 358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.839906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.0.0.1:60588)
0806 01:07:42.850061 (+ 10155us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.850086 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.850252 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.850708 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.850716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.851638 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.852313 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.852326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.854355 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.854360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.856023 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.856044 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.857284 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.857291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.857462 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.858108 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.858130 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.859401 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.864120 (+ 4719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.864123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.864126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.864672 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.865102 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.865105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.865108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.865262 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.865880 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.865885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.866072 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.867342 (+ 1270us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.867625 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9898,"thread_start_us":9626,"threads_started":1}
I20260806 01:07:42.872943 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 31849
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.126@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
I20260806 01:07:42.883301 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.866209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.0.0.1:60604)
0806 01:07:42.866535 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:42.866560 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:42.866693 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:42.867783 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:42.867789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:42.868137 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:42.868304 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.868312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.870333 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.870339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.871271 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:42.871282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.872546 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.872558 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:42.872714 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.873291 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:42.873308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:42.876175 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:42.880445 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.880450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.880454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.880895 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.881356 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:42.881361 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:42.881365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:42.881496 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:42.882050 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:42.882055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:42.882234 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 01:07:42.882399 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 01:07:42.882801 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260806 01:07:42.884377 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.866262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60604 (local address 127.22.63.124:38357)
0806 01:07:42.867348 (+ 1086us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.867353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.867373 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:07:42.867458 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.867461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.867519 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.868419 (+ 900us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.868824 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.870120 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.871414 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.872325 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.872995 (+ 670us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.876402 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.876407 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.876410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.876445 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.880308 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.881049 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.881054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.881063 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.881242 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.881605 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.881610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.881613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.881930 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.882081 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.883638 (+ 1557us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.883895 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":785,"threads_started":1}
I20260806 01:07:42.885964 31752 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.63.124:38357: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
member_type: NON_VOTER
last_known_addr {
host: "127.22.63.124"
port: 38357
}
attrs {
promote: true
}
}
dest_uuid: "3a3b57e3ad26495ebde108ea7f432867"
cas_config_opid_index: 6
Aug 06 01:07:42 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.65@KRBTEST.COM
I20260806 01:07:42.886848 31752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } }
I20260806 01:07:42.889447 32635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 01:07:42.889609 32758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:42.890118 32559 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 01:07:42.892798 32559 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.893486 32758 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 01:07:42.896561 32556 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:42.896466 32635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.898573 362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:42.874367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.22.63.65:39387)
0806 01:07:42.875069 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:42.875081 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:42.875111 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:07:42.875196 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:42.875206 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:42.875509 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:42.875765 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:42.877479 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.879177 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.881168 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:42.882157 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:42.882289 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:42.889121 (+ 6832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:42.889140 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:42.889161 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:42.889207 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:42.893650 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.894602 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.894611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.894615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.894781 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:42.895307 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:42.895312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:42.895315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:42.896057 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:42.896378 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:42.897839 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0806 01:07:42.898117 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":335,"threads_started":1}
I20260806 01:07:42.899374 32559 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.63.124:38357
I20260806 01:07:42.900038 32557 master.cc:537] Master@127.22.63.124:38357 shutting down...
I20260806 01:07:42.900292 32030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:42.900410 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.900729 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:42.906426 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.906631 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.907019 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.907222 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.918260 32758 raft_consensus.cc:858] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } }
W20260806 01:07:42.918737 32758 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } }
I20260806 01:07:42.948021 32758 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.956668 330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3b57e3ad26495ebde108ea7f432867. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.956926 330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 01:07:42.957250 367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.957572 367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 01:07:42.957913 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:42.958247 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:42.957942 372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: NON_VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: true } } }
I20260806 01:07:42.958441 372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: NON_PARTICIPANT
W20260806 01:07:42.959462 31733 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 01:07:42.961567 32557 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
W20260806 01:07:42.961550 31733 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 -> Peer 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124:38357): Couldn't send request to peer 2d777326282e4cd99dcd7fd6ae52cfae. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 01:07:42.962060 32557 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 01:07:42.962242 32557 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: stopping tablet replica
I20260806 01:07:42.980780 32557 master.cc:552] Master@127.22.63.124:38357 shutdown complete.
I20260806 01:07:42.980890 32557 master_runner.cc:305] Clearing existing system tablet
I20260806 01:07:42.981302 32557 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:42.981418 32557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:42.984562 32557 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 01:07:42.986338 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:42.986802 32557 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:42.987083 32557 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 01:07:42.987185 32557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
format_stamp: "Formatted at 2026-08-06 01:07:42 on dist-test-slave-7f01"
server_key: "47ec1f76d6bd8837e72bd15d30fd1838"
server_key_iv: "cfa381879903b3dd9f0fb2b66240faac"
server_key_version: "encryptionkey@0"
I20260806 01:07:42.992123 32557 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:42.999406 32557 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 01:07:42.999600 32557 log.cc:1196] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 01:07:43.000183 32557 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Deleting consensus metadata
I20260806 01:07:43.000458 32557 master_runner.cc:315] Copying system tablet from 127.22.63.126:45699
I20260806 01:07:43.003242 32557 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: tablet copy: Beginning tablet copy session from remote peer at address 127.22.63.126:45699
I20260806 01:07:43.007092 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 31849
I20260806 01:07:43.022226 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.004186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33086)
0806 01:07:43.005892 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:43.005909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:43.006086 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:43.006480 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:43.006484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:43.006953 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:43.007291 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.007309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.009496 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.009501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.011420 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.011439 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.012484 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.012491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.012721 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.013508 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:43.013538 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:43.014887 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:43.018975 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.018983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.018997 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.019560 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.020067 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.020073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.020077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.020286 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.020979 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:43.020984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:43.021181 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 01:07:43.021335 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:07:43.021700 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1579,"thread_start_us":1398,"threads_started":1}
I20260806 01:07:43.022223 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.004223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.22.63.126:45699)
0806 01:07:43.004476 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.004482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.006008 (+ 1526us) server_negotiation.cc:407] Connection header received
0806 01:07:43.006223 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.006229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.006323 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.006441 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:43.007484 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.008782 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.011590 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.012301 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.012438 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.015105 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:43.015114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:43.015119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:43.015159 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:43.018824 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.019713 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.019721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.019727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.019924 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.020418 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.020425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.020431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.020849 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:43.021011 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.021470 (+ 459us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.021744 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260806 01:07:43.023329 31789 tablet_copy_service.cc:140] P 3a3b57e3ad26495ebde108ea7f432867: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2d777326282e4cd99dcd7fd6ae52cfae ({username='oryx', principal='oryx/127.22.63.124@KRBTEST.COM'} at 127.0.0.1:33086)
I20260806 01:07:43.023506 31789 tablet_copy_service.cc:161] P 3a3b57e3ad26495ebde108ea7f432867: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2d777326282e4cd99dcd7fd6ae52cfae at {username='oryx', principal='oryx/127.22.63.124@KRBTEST.COM'} at 127.0.0.1:33086: session id = 2d777326282e4cd99dcd7fd6ae52cfae-00000000000000000000000000000000
I20260806 01:07:43.027175 31789 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 01:07:43.028990 32557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 01:07:43.032249 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:39387
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:45699
--webserver_port=40971
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:07:43.036412 32557 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: tablet copy: Starting download of 0 data blocks...
I20260806 01:07:43.036739 32557 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: tablet copy: Starting download of 1 WAL segments...
I20260806 01:07:43.039302 32557 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 01:07:43.043913 31789 tablet_copy_service.cc:342] P 3a3b57e3ad26495ebde108ea7f432867: Request end of tablet copy session 2d777326282e4cd99dcd7fd6ae52cfae-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.63.124@KRBTEST.COM'} at 127.0.0.1:33086
I20260806 01:07:43.044070 31789 tablet_copy_service.cc:433] P 3a3b57e3ad26495ebde108ea7f432867: ending tablet copy session 2d777326282e4cd99dcd7fd6ae52cfae-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2d777326282e4cd99dcd7fd6ae52cfae
I20260806 01:07:43.057737 32557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:43.060523 389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:43.066030 390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:43.070458 394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:43.073308 32557 server_base.cc:1034] running on GCE node
I20260806 01:07:43.073645 32557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:43.074347 32557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:43.075547 32557 hybrid_clock.cc:648] HybridClock initialized: now 1785978463075493 us; error 42 us; skew 500 ppm
I20260806 01:07:43.075831 32557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978463, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:43.078094 32557 init.cc:377] Logged in from keytab as oryx/127.22.63.124@KRBTEST.COM (short username oryx)
I20260806 01:07:43.084949 32557 webserver.cc:460] Webserver started at http://127.22.63.124:39411/ using document root <none> and password file <none>
I20260806 01:07:43.085304 32557 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:43.085413 32557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:43.088752 32557 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 01:07:43.090782 402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:43.091254 32557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 01:07:43.091387 32557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
format_stamp: "Formatted at 2026-08-06 01:07:42 on dist-test-slave-7f01"
server_key: "47ec1f76d6bd8837e72bd15d30fd1838"
server_key_iv: "cfa381879903b3dd9f0fb2b66240faac"
server_key_version: "encryptionkey@0"
I20260806 01:07:43.091575 32557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:43.105403 32557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:43.110025 32557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:43.137674 32557 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.124:38357
I20260806 01:07:43.137738 471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.124:38357 every 8 connection(s)
I20260806 01:07:43.138907 32557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
I20260806 01:07:43.147536 472 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:07:43.151171 472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Bootstrap starting.
I20260806 01:07:43.167439 472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:07:43.168064 472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: Bootstrap complete.
I20260806 01:07:43.169307 472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } }
I20260806 01:07:43.169660 472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2d777326282e4cd99dcd7fd6ae52cfae, State: Initialized, Role: LEARNER
I20260806 01:07:43.169932 472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } }
I20260806 01:07:43.170946 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:43.171280 472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:43.171471 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:43.195894 491 catalog_manager.cc:1259] Loaded cluster ID: 86921016eb8a44149bd72d50d420a6a2
I20260806 01:07:43.196022 491 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: loading cluster ID for follower catalog manager: success
I20260806 01:07:43.199819 491 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: acquiring CA information for follower catalog manager: success
I20260806 01:07:43.202446 491 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:07:43.363653 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:43.364040 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:43.370155 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.363954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.0.0.1:60610)
0806 01:07:43.364356 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:43.364380 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:43.364522 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:43.365148 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:43.365153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:43.365183 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:43.365387 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.365396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.366786 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.366792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.368088 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.368101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.369306 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.369312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.369547 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.369588 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:07:43.369705 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:07:43.369848 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260806 01:07:43.370208 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.364163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.22.63.124:38357)
0806 01:07:43.364835 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.364840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.364856 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:07:43.364915 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.364919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.364986 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.365120 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:43.365565 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.366613 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.368250 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.369116 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.369273 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.369519 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.369735 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.369874 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":398,"threads_started":1}
I20260806 01:07:43.371285 437 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260806 01:07:43.371858 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a3b57e3ad26495ebde108ea7f432867. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: NON_VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: true } } }
I20260806 01:07:43.372167 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: LEARNER
I20260806 01:07:43.372560 32559 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: attempting to promote NON_VOTER 2d777326282e4cd99dcd7fd6ae52cfae to VOTER
I20260806 01:07:43.373183 32559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:43.374984 437 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 LEARNER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:07:43.375026 32635 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Refusing update from remote peer 3a3b57e3ad26495ebde108ea7f432867: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 01:07:43.375654 32556 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:07:43.376204 32555 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 01:07:43.378077 32556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.378743 32635 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.382218 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.382514 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:43.383504 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.383836 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:43.382239 32556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.383961 32556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:43.387194 32555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.387457 32555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:43.387861 437 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.390689 477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a3b57e3ad26495ebde108ea7f432867" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:43.390980 477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: FOLLOWER
W20260806 01:07:43.416973 388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:43.417302 388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:43.417479 388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:43.425249 388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:43.425400 388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:43.425567 388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:43.435880 388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:39387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=40971
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:43.437469 388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:43.438830 388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:43.453265 502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:43.453243 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:43.453539 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:43.454267 388 server_base.cc:1034] running on GCE node
I20260806 01:07:43.454900 388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:43.456033 388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:43.457247 388 hybrid_clock.cc:648] HybridClock initialized: now 1785978463457198 us; error 47 us; skew 500 ppm
I20260806 01:07:43.457593 388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978463, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:43.460871 388 init.cc:377] Logged in from keytab as oryx/127.22.63.65@KRBTEST.COM (short username oryx)
I20260806 01:07:43.462396 388 webserver.cc:460] Webserver started at http://127.22.63.65:40971/ using document root <none> and password file <none>
I20260806 01:07:43.463205 388 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:43.463372 388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:43.468998 388 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:07:43.472517 513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:43.473944 388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:07:43.474094 388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "345b5bd3e2d745f399caa6fd83dc1670"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "bac7735471f6fb72c556f801c46060a1"
server_key_iv: "14bb1ca565dd341b9521b503367c4a06"
server_key_version: "encryptionkey@0"
I20260806 01:07:43.475117 388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:43.499797 388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:43.504189 388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:43.504531 388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:43.505565 388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:43.507017 388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:43.507112 388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:43.507242 388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:43.507349 388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:43.542485 388 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39387
I20260806 01:07:43.542585 671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39387 every 8 connection(s)
I20260806 01:07:43.544109 388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:43.553596 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 388
I20260806 01:07:43.553953 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 32050
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978463, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978463, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978463, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
Aug 06 01:07:43 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.66@KRBTEST.COM
I20260806 01:07:43.583804 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.547525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:52535 (local address 127.22.63.126:45699)
0806 01:07:43.561859 (+ 14334us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.561870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.561898 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:07:43.561982 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.561990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.562100 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.562249 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:43.564466 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.565653 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.569619 (+ 3966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.570307 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.570463 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.575809 (+ 5346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:43.575816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:43.575821 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:43.575866 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:43.579499 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.580669 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.580675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.580678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.580847 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.581450 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.581454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.581458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.581821 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:43.581962 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.583125 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.583372 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14136,"thread_start_us":14919,"threads_started":1}
I20260806 01:07:43.583811 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.547212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:36821 (local address 127.22.63.125:35243)
0806 01:07:43.547810 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.547816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.554002 (+ 6186us) server_negotiation.cc:407] Connection header received
0806 01:07:43.554303 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.554313 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.554423 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.554526 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:43.566021 (+ 11495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.567245 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.569290 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.569982 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.570129 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.575448 (+ 5319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:43.575457 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:43.575463 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:43.575506 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:43.579425 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.580691 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.580696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.580699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.580854 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.581447 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.581452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.581456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.581833 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:43.581961 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.583123 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.583360 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":411,"threads_started":1}
I20260806 01:07:43.585062 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.550079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:49345 (local address 127.22.63.124:38357)
0806 01:07:43.550291 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.550300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.562250 (+ 11950us) server_negotiation.cc:407] Connection header received
0806 01:07:43.562769 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.562775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.562867 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.562941 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:43.564607 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.565794 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.568567 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.569249 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.569341 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.575907 (+ 6566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:43.575914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:43.575918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:43.575966 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:43.580815 (+ 4849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.581816 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.581822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.581825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.582253 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.582809 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.582816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.582820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.583223 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:43.583374 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.584437 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.584680 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:07:43.586354 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.547794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:52535)
0806 01:07:43.548989 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:43.549127 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:43.552907 (+ 3780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:43.562285 (+ 9378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:43.562290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:43.562934 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:43.564192 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.564269 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.565814 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.565829 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.568415 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.568432 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.570527 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.570533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.570734 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.571312 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:43.571350 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:43.575537 (+ 4187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:43.579633 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.579637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.579641 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.580498 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.580992 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.580997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.581001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.581297 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.582041 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:43.582050 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:43.582636 (+ 586us) client_negotiation.cc:769] Sending connection context
0806 01:07:43.584048 (+ 1412us) client_negotiation.cc:240] Negotiation successful
0806 01:07:43.584847 (+ 799us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":387,"spinlock_wait_cycles":23296,"thread_start_us":14345,"threads_started":1}
I20260806 01:07:43.586354 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.547795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.22.63.65:36821)
0806 01:07:43.553865 (+ 6070us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:43.553903 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:43.554133 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:43.561414 (+ 7281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:43.561432 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:43.564454 (+ 3022us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:43.564796 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.564806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.567421 (+ 2615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.567426 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.569091 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.569106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.570161 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.570174 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.570450 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.571035 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:43.571091 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:43.575148 (+ 4057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:43.579601 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.579612 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.579631 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.580561 (+ 930us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.580996 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.581000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.581003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.581297 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.582041 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:43.582049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:43.582636 (+ 587us) client_negotiation.cc:769] Sending connection context
0806 01:07:43.584140 (+ 1504us) client_negotiation.cc:240] Negotiation successful
0806 01:07:43.584727 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5865,"mutex_wait_us":180,"spinlock_wait_cycles":23424,"thread_start_us":462,"threads_started":1}
I20260806 01:07:43.587265 679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.549642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.22.63.65:49345)
0806 01:07:43.562124 (+ 12482us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:43.562152 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:43.562598 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:43.563166 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:43.563172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:43.563767 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:43.564443 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.564454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.566337 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.566342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.568407 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:43.568423 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.569722 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.569730 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:43.570025 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.571076 (+ 1051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:43.571104 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:43.575615 (+ 4511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:43.581107 (+ 5492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.581112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.581115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.581670 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.582439 (+ 769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:43.582445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:43.582449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:43.582672 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:43.583488 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:43.583492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:43.583656 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 01:07:43.585636 (+ 1980us) client_negotiation.cc:240] Negotiation successful
0806 01:07:43.585928 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12327,"mutex_wait_us":348,"spinlock_wait_cycles":3840,"thread_start_us":3864,"threads_started":1}
I20260806 01:07:43.588158 673 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:43.588244 675 heartbeater.cc:347] Connected to a master server at 127.22.63.125:35243
I20260806 01:07:43.588532 675 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:43.588541 673 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:43.589108 674 heartbeater.cc:347] Connected to a master server at 127.22.63.124:38357
I20260806 01:07:43.589255 674 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:43.589641 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:43.555316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.22.63.66:46621)
0806 01:07:43.565442 (+ 10126us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:43.565451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:43.565480 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:07:43.565545 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:43.565554 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:43.565785 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:43.565945 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:43.568031 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.569436 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.572700 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:43.573465 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:43.573616 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:43.581109 (+ 7493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:43.581125 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:43.581144 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:43.581191 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:43.585291 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.586227 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.586238 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.586244 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.586414 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:43.586941 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:43.586950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:43.586956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:43.587411 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:43.587717 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:43.588943 (+ 1226us) server_negotiation.cc:299] Negotiation successful
0806 01:07:43.589346 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9920,"thread_start_us":9737,"threads_started":1}
I20260806 01:07:43.589681 675 heartbeater.cc:511] Master 127.22.63.125:35243 requested a full tablet report, sending...
I20260806 01:07:43.589769 674 heartbeater.cc:511] Master 127.22.63.124:38357 requested a full tablet report, sending...
I20260806 01:07:43.590590 673 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:43.591492 32227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:43.592527 32618 ts_manager.cc:194] Registered new tserver with Master: 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:43.592549 426 ts_manager.cc:194] Registered new tserver with Master: 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:43.594175 31752 ts_manager.cc:194] Re-registered known tserver with Master: 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:43.595513 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:52535
I20260806 01:07:43.676573 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 32050
I20260806 01:07:43.702457 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:46621
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:45699
--webserver_port=38353
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:44.056329 685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:44.056620 685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:44.056767 685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:44.064164 685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:44.064327 685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:44.064496 685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:44.074586 685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:46621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=38353
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:44.076049 685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:44.077380 685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:44.092244 691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:44.092213 692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:44.092204 695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:44.093144 685 server_base.cc:1034] running on GCE node
I20260806 01:07:44.093837 685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:44.094775 685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:44.095969 685 hybrid_clock.cc:648] HybridClock initialized: now 1785978464095905 us; error 55 us; skew 500 ppm
I20260806 01:07:44.096284 685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:44.099473 685 init.cc:377] Logged in from keytab as oryx/127.22.63.66@KRBTEST.COM (short username oryx)
I20260806 01:07:44.101006 685 webserver.cc:460] Webserver started at http://127.22.63.66:38353/ using document root <none> and password file <none>
I20260806 01:07:44.101708 685 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:44.101864 685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:44.107189 685 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:07:44.110293 709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.111864 685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:07:44.112043 685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "84087abafdfd4311b171b1e151889812"
format_stamp: "Formatted at 2026-08-06 01:07:40 on dist-test-slave-7f01"
server_key: "3c23d7164b97db0a242f255bdf4db048"
server_key_iv: "98253caef1af1e8cf57399f5ceb445d2"
server_key_version: "encryptionkey@0"
I20260806 01:07:44.113034 685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:44.128688 685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:44.133188 685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:44.133493 685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:44.134529 685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:44.136008 685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:44.136127 685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.136256 685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:44.136348 685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.171166 685 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:46621
I20260806 01:07:44.171281 864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:46621 every 8 connection(s)
I20260806 01:07:44.172734 685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:44.180341 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 685
I20260806 01:07:44.180684 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 32243
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
I20260806 01:07:44.210012 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.175657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:56081 (local address 127.22.63.126:45699)
0806 01:07:44.176276 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.176286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.178004 (+ 1718us) server_negotiation.cc:407] Connection header received
0806 01:07:44.185405 (+ 7401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.185416 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.185543 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.185703 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.189888 (+ 4185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.191383 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.194890 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.195560 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.195691 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.202395 (+ 6704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.202400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.202403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.202441 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.205863 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.206633 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.206637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.206639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.206774 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.207278 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.207287 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.207292 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.207806 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.207982 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.208960 (+ 978us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.209378 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":283,"threads_started":1}
I20260806 01:07:44.210971 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.176502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:56081)
0806 01:07:44.177769 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.177841 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.184977 (+ 7136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.187652 (+ 2675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.187669 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.188668 (+ 999us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.189669 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.189699 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.192531 (+ 2832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.192539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.194731 (+ 2192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.194750 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.197510 (+ 2760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.197520 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.197851 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.199586 (+ 1735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.199615 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.202159 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.206073 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.206080 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.206094 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.206501 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.206894 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.206899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.206903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.207128 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.207984 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.207994 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.208495 (+ 501us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.208826 (+ 331us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.209391 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":787,"threads_started":1}
I20260806 01:07:44.212724 865 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:44.213090 865 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.214244 877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.177196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.22.63.66:57133)
0806 01:07:44.180936 (+ 3740us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.180962 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.193514 (+ 12552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.194171 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.194179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.194938 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.195240 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.195253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.196930 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.196938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.199000 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.199020 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.200538 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.200547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.200804 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.201466 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.201494 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.204599 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.209563 (+ 4964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.209567 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.209571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.210450 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.210923 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.210927 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.210930 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.211150 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.212488 (+ 1338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.212492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.212654 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.212823 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.213130 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3563,"thread_start_us":2590,"threads_started":1}
I20260806 01:07:44.214278 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.175645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:57133 (local address 127.22.63.125:35243)
0806 01:07:44.192503 (+ 16858us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.192512 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.192538 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:07:44.193662 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.193671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.193791 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.193920 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.195438 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.196716 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.199222 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.200082 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.200221 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.204836 (+ 4615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.204842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.204847 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.204907 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.209391 (+ 4484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.210583 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.210592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.210598 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.210767 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.211320 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.211342 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.211348 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.212206 (+ 858us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.212405 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.213728 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.213957 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16659,"thread_start_us":16901,"threads_started":1}
I20260806 01:07:44.214581 865 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:44.215094 867 heartbeater.cc:347] Connected to a master server at 127.22.63.125:35243
I20260806 01:07:44.215211 867 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.215642 867 heartbeater.cc:511] Master 127.22.63.125:35243 requested a full tablet report, sending...
I20260806 01:07:44.216428 31752 ts_manager.cc:194] Re-registered known tserver with Master: 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:44.216879 32618 ts_manager.cc:194] Registered new tserver with Master: 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:44.217562 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:56081
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.67@KRBTEST.COM
I20260806 01:07:44.229014 879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.180754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:52133 (local address 127.22.63.124:38357)
0806 01:07:44.212613 (+ 31859us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.212620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.212639 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:07:44.212701 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.212705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.212790 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.213901 (+ 1111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.216445 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.217561 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.219156 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.219597 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.219731 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.223145 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.223150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.223153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.223185 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.226301 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.226943 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.226949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.226957 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.227121 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.227596 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.227599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.227601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.227968 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.228083 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.228537 (+ 454us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.228704 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31719,"thread_start_us":31600,"threads_started":1}
I20260806 01:07:44.229230 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.180360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.22.63.66:52133)
0806 01:07:44.199327 (+ 18967us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.199346 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.199467 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.213039 (+ 13572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.213047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.214171 (+ 1124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.216225 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.216239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.217736 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.217741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.219001 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.219014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.219850 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.219857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.220043 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.220641 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.220656 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.222945 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.226443 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.226448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.226452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.226815 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.227260 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.227268 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.227274 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.227473 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.228092 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.228100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.228272 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.228428 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.228775 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18828,"mutex_wait_us":28,"thread_start_us":18687,"threads_started":1}
I20260806 01:07:44.230100 866 heartbeater.cc:347] Connected to a master server at 127.22.63.124:38357
I20260806 01:07:44.230260 866 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.230706 866 heartbeater.cc:511] Master 127.22.63.124:38357 requested a full tablet report, sending...
I20260806 01:07:44.231948 426 ts_manager.cc:194] Registered new tserver with Master: 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:44.233119 883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.181906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.22.63.67:39683)
0806 01:07:44.211459 (+ 29553us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.211469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.211506 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:07:44.211580 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.211591 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.211859 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.212034 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.215034 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.218821 (+ 3787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.220710 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.221272 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.221426 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.226381 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.226394 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.226408 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.226455 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.230128 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.230890 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.230902 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.230908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.231079 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.231551 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.231555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.231557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.231974 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.232299 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.232623 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.232828 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28999,"thread_start_us":28809,"threads_started":1}
I20260806 01:07:44.234462 32409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:44.322518 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 32243
I20260806 01:07:44.345867 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:39683
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:45699
--webserver_port=46125
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:41833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:07:44.598186 673 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
W20260806 01:07:44.704336 887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:44.704664 887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:44.704839 887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:44.712445 887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:44.712594 887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:44.712764 887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:44.722916 887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:41833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:39683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=46125
--tserver_master_addrs=127.22.63.126:45699,127.22.63.125:35243,127.22.63.124:38357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:44.724416 887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:44.725751 887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:44.739723 895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:44.739861 894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:44.739880 899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:44.741093 887 server_base.cc:1034] running on GCE node
I20260806 01:07:44.741748 887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:44.742853 887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:44.744088 887 hybrid_clock.cc:648] HybridClock initialized: now 1785978464744038 us; error 45 us; skew 500 ppm
I20260806 01:07:44.744429 887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:44.747956 887 init.cc:377] Logged in from keytab as oryx/127.22.63.67@KRBTEST.COM (short username oryx)
I20260806 01:07:44.749526 887 webserver.cc:460] Webserver started at http://127.22.63.67:46125/ using document root <none> and password file <none>
I20260806 01:07:44.750241 887 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:44.750406 887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:44.755954 887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:07:44.758912 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.760473 887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:44.760654 887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
format_stamp: "Formatted at 2026-08-06 01:07:41 on dist-test-slave-7f01"
server_key: "3fd347473895fd7a8c40ed44ffcf05e4"
server_key_iv: "e37ccc626a10ca32c4b89005ba5724a0"
server_key_version: "encryptionkey@0"
I20260806 01:07:44.761668 887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:44.780263 887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:44.784968 887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:44.785272 887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:44.786311 887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:07:44.787781 887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:44.787906 887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.788043 887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:44.788144 887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:44.822943 887 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:39683
I20260806 01:07:44.822992 1072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:39683 every 8 connection(s)
I20260806 01:07:44.824611 887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:44.834551 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 887
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.124@KRBTEST.COM
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.125@KRBTEST.COM
Aug 06 01:07:44 dist-test-slave-7f01 krb5kdc[31690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978464, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:44.857088 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.828259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:35605 (local address 127.22.63.124:38357)
0806 01:07:44.828963 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.828967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.829852 (+ 885us) server_negotiation.cc:407] Connection header received
0806 01:07:44.832813 (+ 2961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.832820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.832909 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.832983 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.835861 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.836788 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.839463 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.840018 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.840158 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.846567 (+ 6409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.846574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.846579 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.846646 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.852654 (+ 6008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.853700 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.853705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.853709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.853859 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.854491 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.854496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.854499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.854882 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.855051 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.856269 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.856523 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":476,"threads_started":1}
I20260806 01:07:44.859369 1087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.830256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.22.63.67:41045)
0806 01:07:44.830878 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.830899 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.832517 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.833264 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.833271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.834348 (+ 1077us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.835620 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.835650 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.837428 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.837433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.839088 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.839106 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.840448 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.840457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.840634 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.841725 (+ 1091us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.841770 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.846942 (+ 5172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.854245 (+ 7303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.854250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.854254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.854879 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.855463 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.855468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.855472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.855669 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.856403 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.856410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.856577 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.856747 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.857506 (+ 759us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":263,"thread_start_us":338,"threads_started":1}
I20260806 01:07:44.859889 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.828592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.22.63.67:35605)
0806 01:07:44.829665 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.829726 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.832510 (+ 2784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.833262 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.833269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.834347 (+ 1078us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.835619 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.835647 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.836987 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.836991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.839335 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.839343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.840260 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.840270 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.840567 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.841432 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.841472 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.846160 (+ 4688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.852813 (+ 6653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.852823 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.852842 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.853287 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.854022 (+ 735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.854027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.854030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.854326 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.855164 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.855175 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.855775 (+ 600us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.857373 (+ 1598us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.857771 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":627,"spinlock_wait_cycles":27264,"thread_start_us":598,"threads_started":1}
I20260806 01:07:44.860277 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.830410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:41045 (local address 127.22.63.125:35243)
0806 01:07:44.832070 (+ 1660us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.832074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.832091 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:07:44.832792 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.832797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.832898 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.833021 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.835872 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.837256 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.839257 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.840205 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.844545 (+ 4340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.847200 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.847210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.847218 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.847273 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.853996 (+ 6723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.855078 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.855084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.855087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.855265 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.855856 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.855860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.855863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.856238 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.859364 (+ 3126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.859567 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.859841 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1562,"thread_start_us":1454,"threads_started":1}
I20260806 01:07:44.861653 1076 heartbeater.cc:347] Connected to a master server at 127.22.63.124:38357
I20260806 01:07:44.861690 1077 heartbeater.cc:347] Connected to a master server at 127.22.63.125:35243
I20260806 01:07:44.862087 1077 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.862116 1076 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.863442 1077 heartbeater.cc:511] Master 127.22.63.125:35243 requested a full tablet report, sending...
I20260806 01:07:44.863492 1076 heartbeater.cc:511] Master 127.22.63.124:38357 requested a full tablet report, sending...
I20260806 01:07:44.865947 426 ts_manager.cc:194] Registered new tserver with Master: c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:44.866328 32618 ts_manager.cc:194] Registered new tserver with Master: c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:44.869275 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.827758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:40867 (local address 127.22.63.126:45699)
0806 01:07:44.828462 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.828469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.830783 (+ 2314us) server_negotiation.cc:407] Connection header received
0806 01:07:44.833228 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.833233 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.833315 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.833440 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.840156 (+ 6716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.842058 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.844477 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.845246 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.845430 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.852154 (+ 6724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.852162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.852169 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.852221 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.865044 (+ 12823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.866899 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.866905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.866909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.867090 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.867548 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.867556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.867561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.868131 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.868292 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.868686 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.868918 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":386,"threads_started":1}
I20260806 01:07:44.869642 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.828591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:40867)
0806 01:07:44.829665 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.829725 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.832510 (+ 2785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.833490 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.833497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.834347 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:44.835619 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.835647 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.842229 (+ 6582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.842234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.844285 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.844302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.845568 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.845577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.845855 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.847419 (+ 1564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:44.847453 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:44.851910 (+ 4457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:44.865227 (+ 13317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.865234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.865238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.865960 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.867249 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:44.867254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:44.867256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:44.867399 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:44.868253 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:44.868258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:44.868417 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.868566 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.868890 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":173,"spinlock_wait_cycles":17280,"thread_start_us":415,"threads_started":1}
I20260806 01:07:44.870682 1073 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45699
I20260806 01:07:44.870834 1073 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:44.871222 1073 heartbeater.cc:511] Master 127.22.63.126:45699 requested a full tablet report, sending...
I20260806 01:07:44.872375 31752 ts_manager.cc:194] Re-registered known tserver with Master: c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:44.873505 31752 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:40867
I20260806 01:07:44.887498 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.859091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.22.63.125:35243)
0806 01:07:44.861246 (+ 2155us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.861254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.869511 (+ 8257us) server_negotiation.cc:407] Connection header received
0806 01:07:44.869803 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.869811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.869960 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.870041 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.873351 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.874765 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.876484 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.876956 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.877084 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.880830 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.880835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.880838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.880871 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.884200 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.884902 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.884907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.884911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.885047 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.885418 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.885423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.885427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.885702 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.885983 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.886880 (+ 897us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.887119 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2016,"thread_start_us":9615,"threads_started":1}
I20260806 01:07:44.890362 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.856481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.22.63.126:45699)
0806 01:07:44.869547 (+ 13066us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.869552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.869568 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:07:44.869612 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.869617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.869712 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.869834 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.872184 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.873348 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.877339 (+ 3991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.878363 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.878496 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.882124 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.882131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.882136 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.882177 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.886035 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.886890 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.886897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.886902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.887062 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.888873 (+ 1811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.888878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.888881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.889235 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.889456 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.889839 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.890021 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12959,"thread_start_us":15454,"threads_started":1}
I20260806 01:07:44.893354 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.872554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.22.63.124:38357)
0806 01:07:44.872744 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.872751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.876958 (+ 4207us) server_negotiation.cc:407] Connection header received
0806 01:07:44.877379 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.877385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.877463 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.877575 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:44.880489 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.881725 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.883144 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.883855 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.884011 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.887209 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:44.887216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:44.887221 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:44.887261 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:44.890430 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.891211 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.891215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.891217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.891344 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:44.891826 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:44.891831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:44.891834 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:44.892130 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:44.892442 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.892640 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.892950 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:07:44.894845 31752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:44.901713 31752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:44.939617 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.920759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.22.63.66:46621)
0806 01:07:44.931325 (+ 10566us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.931331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.931354 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:07:44.931505 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.931512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.931722 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.931958 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:44.932947 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.935049 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.937338 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.938550 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.938698 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.938940 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.939153 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.939320 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10421,"thread_start_us":10310,"threads_started":1}
I20260806 01:07:44.939662 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.920463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:46621 (local address 127.0.0.1:35176)
0806 01:07:44.921887 (+ 1424us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.921907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.922040 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.931999 (+ 9959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.932011 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.932052 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:44.932740 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.932754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.935190 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.935196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.937124 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.937141 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.938745 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.938754 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.938941 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.938982 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.939085 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.939218 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"mutex_wait_us":26,"thread_start_us":11828,"threads_started":1}
I20260806 01:07:44.940982 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.920989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39683 (local address 127.0.0.1:36822)
0806 01:07:44.929486 (+ 8497us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.929511 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.929682 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.931856 (+ 2174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.931863 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.931898 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:44.932538 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.932553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.934730 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.934735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.937255 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.937276 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.940190 (+ 2914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.940195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.940315 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.940338 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.940433 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.940580 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8320,"mutex_wait_us":36,"thread_start_us":8135,"threads_started":1}
I20260806 01:07:44.941885 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.921682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.22.63.67:39683)
0806 01:07:44.928905 (+ 7223us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.928913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.929843 (+ 930us) server_negotiation.cc:407] Connection header received
0806 01:07:44.929985 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.929998 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.930357 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.930621 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:44.932747 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.934530 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.937483 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.939401 (+ 1918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.939554 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.941310 (+ 1756us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.941385 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.941559 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7007,"thread_start_us":6772,"threads_started":1}
I20260806 01:07:44.942067 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.921864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.22.63.65:39387)
0806 01:07:44.928245 (+ 6381us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:44.928258 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:44.932803 (+ 4545us) server_negotiation.cc:407] Connection header received
0806 01:07:44.933139 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:44.933146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:44.933381 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:44.933559 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:44.934499 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.936116 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.938619 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.940322 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.940464 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.940703 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:44.940893 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 01:07:44.941371 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6192,"thread_start_us":6069,"threads_started":1}
I20260806 01:07:44.943398 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:44.921313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:39387 (local address 127.0.0.1:40974)
0806 01:07:44.932677 (+ 11364us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:44.932699 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:44.932923 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:44.933829 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:44.933833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:44.933856 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:44.934241 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.934256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.936290 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.936299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.938449 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:44.938467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:44.940532 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:44.940538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:44.940711 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:44.940743 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:07:44.940841 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:07:44.943084 (+ 2243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11198,"thread_start_us":3034,"threads_started":1}
I20260806 01:07:44.944411 781 tablet_service.cc:1467] Processing CreateTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 (DEFAULT_TABLE table=test-table [id=e05781ba80f040f0a833f8ee422bf448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:44.945734 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d5cc47efe2482f8e7956e66e391c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:44.945384 981 tablet_service.cc:1467] Processing CreateTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 (DEFAULT_TABLE table=test-table [id=e05781ba80f040f0a833f8ee422bf448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:44.946755 981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d5cc47efe2482f8e7956e66e391c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:44.947074 585 tablet_service.cc:1467] Processing CreateTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 (DEFAULT_TABLE table=test-table [id=e05781ba80f040f0a833f8ee422bf448]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:44.948395 585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d5cc47efe2482f8e7956e66e391c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:44.958324 1124 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Bootstrap starting.
I20260806 01:07:44.962309 1128 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Bootstrap starting.
I20260806 01:07:44.962745 1124 tablet_bootstrap.cc:654] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:44.964239 1124 log.cc:824] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:44.966379 1128 tablet_bootstrap.cc:654] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:44.967671 1128 log.cc:824] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:44.967947 1131 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Bootstrap starting.
I20260806 01:07:44.968067 1124 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: No bootstrap required, opened a new log
I20260806 01:07:44.968369 1124 ts_tablet_manager.cc:1397] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 01:07:44.971239 1131 tablet_bootstrap.cc:654] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:44.972383 1131 log.cc:824] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:44.974946 1131 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: No bootstrap required, opened a new log
I20260806 01:07:44.975215 1131 ts_tablet_manager.cc:1397] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260806 01:07:44.978690 1128 tablet_bootstrap.cc:492] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: No bootstrap required, opened a new log
I20260806 01:07:44.979000 1128 ts_tablet_manager.cc:1397] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20260806 01:07:44.981134 1124 raft_consensus.cc:348] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.981632 1124 raft_consensus.cc:374] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:44.981745 1124 raft_consensus.cc:729] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84087abafdfd4311b171b1e151889812, State: Initialized, Role: FOLLOWER
I20260806 01:07:44.982386 1124 consensus_queue.cc:260] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.983526 865 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
I20260806 01:07:44.984531 1124 ts_tablet_manager.cc:1428] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260806 01:07:44.984311 1131 raft_consensus.cc:348] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.984732 1131 raft_consensus.cc:374] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:44.984826 1131 raft_consensus.cc:729] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345b5bd3e2d745f399caa6fd83dc1670, State: Initialized, Role: FOLLOWER
I20260806 01:07:44.985507 1131 consensus_queue.cc:260] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.989530 1128 raft_consensus.cc:348] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.990175 1128 raft_consensus.cc:374] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:44.990319 1128 raft_consensus.cc:729] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cbbbcdfe5d450fb40a1dbcaf1d7f28, State: Initialized, Role: FOLLOWER
I20260806 01:07:44.990959 1128 consensus_queue.cc:260] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:44.991524 1131 ts_tablet_manager.cc:1428] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Time spent starting tablet: real 0.016s user 0.018s sys 0.000s
I20260806 01:07:44.996078 1073 heartbeater.cc:503] Master 127.22.63.126:45699 was elected leader, sending a full tablet report...
I20260806 01:07:44.996863 1128 ts_tablet_manager.cc:1428] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Time spent starting tablet: real 0.018s user 0.014s sys 0.005s
I20260806 01:07:45.035223 1137 raft_consensus.cc:482] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:45.035580 1137 raft_consensus.cc:504] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:45.037448 1137 leader_election.cc:288] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683), 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:45.046432 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.037729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:36053 (local address 127.22.63.67:39683)
0806 01:07:45.038075 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.038083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.038551 (+ 468us) server_negotiation.cc:407] Connection header received
0806 01:07:45.038765 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.038769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.038864 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.038969 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:45.039663 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.040877 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.043978 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.045219 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.045401 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.045478 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.046010 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.046149 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260806 01:07:45.046416 1123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.037999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:46505 (local address 127.22.63.65:39387)
0806 01:07:45.038262 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.038270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.038859 (+ 589us) server_negotiation.cc:407] Connection header received
0806 01:07:45.038950 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.038956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.039057 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.039151 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:45.039794 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.040897 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.044001 (+ 3104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.045237 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.045393 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.045473 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.046010 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.046149 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:07:45.046741 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.037598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39683 (local address 127.22.63.66:36053)
0806 01:07:45.038409 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:45.038437 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:45.038620 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:45.039012 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:45.039016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:45.039049 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:45.039504 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:45.039522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.041022 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.041027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:45.043776 (+ 2749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:45.043791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.045472 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.045480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:45.045784 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.045869 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 01:07:45.045960 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 01:07:45.046239 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":346,"thread_start_us":836,"threads_started":1}
I20260806 01:07:45.046775 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.037722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:39387 (local address 127.22.63.66:46505)
0806 01:07:45.038730 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:45.038747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:45.038849 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:45.039165 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:45.039171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:45.039198 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:45.039590 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:45.039601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.041023 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.041027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:45.043774 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:45.043791 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.045486 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.045490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:45.045779 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.045861 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 01:07:45.045960 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:07:45.046239 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"mutex_wait_us":281,"thread_start_us":376,"threads_started":1}
W20260806 01:07:45.047524 676 tablet.cc:2369] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:45.047921 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d5cc47efe2482f8e7956e66e391c22" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" is_pre_election: true
I20260806 01:07:45.048040 614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d5cc47efe2482f8e7956e66e391c22" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345b5bd3e2d745f399caa6fd83dc1670" is_pre_election: true
I20260806 01:07:45.048508 1007 raft_consensus.cc:2454] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 0.
I20260806 01:07:45.048590 614 raft_consensus.cc:2454] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 0.
I20260806 01:07:45.049201 711 leader_election.cc:302] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84087abafdfd4311b171b1e151889812, c8cbbbcdfe5d450fb40a1dbcaf1d7f28; no voters:
I20260806 01:07:45.049690 1137 raft_consensus.cc:2781] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:45.049840 1137 raft_consensus.cc:482] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:45.049968 1137 raft_consensus.cc:3037] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:45.052309 1137 raft_consensus.cc:504] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:45.052939 1137 leader_election.cc:288] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 election: Requested vote from peers c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683), 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387)
I20260806 01:07:45.053381 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d5cc47efe2482f8e7956e66e391c22" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
I20260806 01:07:45.053416 614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d5cc47efe2482f8e7956e66e391c22" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345b5bd3e2d745f399caa6fd83dc1670"
I20260806 01:07:45.053638 1007 raft_consensus.cc:3037] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:45.053632 614 raft_consensus.cc:3037] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:45.057010 1007 raft_consensus.cc:2454] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 1.
I20260806 01:07:45.057021 614 raft_consensus.cc:2454] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 1.
I20260806 01:07:45.057464 711 leader_election.cc:302] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84087abafdfd4311b171b1e151889812, c8cbbbcdfe5d450fb40a1dbcaf1d7f28; no voters:
I20260806 01:07:45.057794 1137 raft_consensus.cc:2781] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:45.058236 1137 raft_consensus.cc:686] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 1 LEADER]: Becoming Leader. State: Replica: 84087abafdfd4311b171b1e151889812, State: Running, Role: LEADER
I20260806 01:07:45.058775 1137 consensus_queue.cc:237] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } }
I20260806 01:07:45.064023 31752 catalog_manager.cc:5495] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84087abafdfd4311b171b1e151889812 (127.22.63.66). New cstate: current_term: 1 leader_uuid: "84087abafdfd4311b171b1e151889812" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } health_report { overall_health: UNKNOWN } } }
W20260806 01:07:45.078991 1079 tablet.cc:2369] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:45.096951 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.091295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35178 (local address 127.22.63.66:46621)
0806 01:07:45.091548 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.091555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.091660 (+ 105us) server_negotiation.cc:407] Connection header received
0806 01:07:45.091937 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.091941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.092057 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.092221 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:45.092612 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.093397 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.094743 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.095160 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.095292 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.095843 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:45.096366 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:45.096535 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.096588 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.096671 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:07:45.105155 1007 raft_consensus.cc:1261] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Refusing update from remote peer 84087abafdfd4311b171b1e151889812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:45.105171 614 raft_consensus.cc:1261] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Refusing update from remote peer 84087abafdfd4311b171b1e151889812: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:45.106148 1144 consensus_queue.cc:1035] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:45.106487 1137 consensus_queue.cc:1035] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:45.138963 31751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:45.139376 31751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:45.148350 31751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a3b57e3ad26495ebde108ea7f432867: Sending DeleteTablet for 3 replicas of tablet e6d5cc47efe2482f8e7956e66e391c22
I20260806 01:07:45.149989 781 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:45 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:35176
I20260806 01:07:45.151141 981 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:45 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:36822
I20260806 01:07:45.151262 585 tablet_service.cc:1514] Processing DeleteTablet for tablet e6d5cc47efe2482f8e7956e66e391c22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:45 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:40974
I20260806 01:07:45.164564 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 31706
I20260806 01:07:45.165740 31801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:45.167269 1164 tablet_replica.cc:330] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: stopping tablet replica
I20260806 01:07:45.167616 1165 tablet_replica.cc:330] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: stopping tablet replica
I20260806 01:07:45.168018 1164 raft_consensus.cc:2229] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:45.168023 1165 raft_consensus.cc:2229] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:45.168582 1165 raft_consensus.cc:2258] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:45.168610 1164 raft_consensus.cc:2258] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:45.169427 1163 tablet_replica.cc:330] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: stopping tablet replica
I20260806 01:07:45.169922 1163 raft_consensus.cc:2229] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:45.171029 1164 ts_tablet_manager.cc:1905] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:45.180310 1163 raft_consensus.cc:2258] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:45.180866 1165 ts_tablet_manager.cc:1905] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:45.190059 1163 ts_tablet_manager.cc:1905] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:45.191932 1164 ts_tablet_manager.cc:1918] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:45.192170 1164 log.cc:1196] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/e6d5cc47efe2482f8e7956e66e391c22
I20260806 01:07:45.192831 1164 ts_tablet_manager.cc:1939] T e6d5cc47efe2482f8e7956e66e391c22 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting consensus metadata
I20260806 01:07:45.198513 1165 ts_tablet_manager.cc:1918] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:45.198724 1165 log.cc:1196] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/e6d5cc47efe2482f8e7956e66e391c22
I20260806 01:07:45.199077 1163 ts_tablet_manager.cc:1918] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:45.199275 1163 log.cc:1196] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/e6d5cc47efe2482f8e7956e66e391c22
I20260806 01:07:45.199476 1165 ts_tablet_manager.cc:1939] T e6d5cc47efe2482f8e7956e66e391c22 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting consensus metadata
I20260806 01:07:45.206634 1163 ts_tablet_manager.cc:1939] T e6d5cc47efe2482f8e7956e66e391c22 P 84087abafdfd4311b171b1e151889812: Deleting consensus metadata
I20260806 01:07:45.220818 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.154443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52278 (local address 127.22.63.125:35243)
0806 01:07:45.154623 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.154630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.161756 (+ 7126us) server_negotiation.cc:407] Connection header received
0806 01:07:45.161830 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.161835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.161931 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.162036 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:45.176772 (+ 14736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.177994 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.180598 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.181336 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.181485 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.202009 (+ 20524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:45.202021 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:45.202027 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:45.202081 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:45.205928 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.206969 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.207005 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.207008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.207208 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.208007 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.208013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.208017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.208427 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:45.208788 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.220030 (+ 11242us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.220368 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:07:45.236090 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.161532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.22.63.124:38357)
0806 01:07:45.161709 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.161715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.217522 (+ 55807us) server_negotiation.cc:407] Connection header received
0806 01:07:45.217901 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.217907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.218002 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.218099 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:45.220139 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.221354 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.224077 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.224913 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.225083 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.228683 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:45.228694 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:45.228701 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:45.228762 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:45.232777 (+ 4015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.233653 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.233660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.233664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.233846 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.234329 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.234335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.234339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.234743 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:45.235030 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.235384 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.235640 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:07:45.335945 31731 catalog_manager.cc:4841] TS c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683): tablet e6d5cc47efe2482f8e7956e66e391c22 (table test-table [id=e05781ba80f040f0a833f8ee422bf448]) successfully deleted
W20260806 01:07:45.336328 31731 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.22.63.67:39683, user_credentials={real_user=slave}} blocked reactor thread for 141337us
I20260806 01:07:45.336588 31733 catalog_manager.cc:4841] TS 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387): tablet e6d5cc47efe2482f8e7956e66e391c22 (table test-table [id=e05781ba80f040f0a833f8ee422bf448]) successfully deleted
I20260806 01:07:45.336709 31730 catalog_manager.cc:4841] TS 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621): tablet e6d5cc47efe2482f8e7956e66e391c22 (table test-table [id=e05781ba80f040f0a833f8ee422bf448]) successfully deleted
W20260806 01:07:45.336776 31733 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.22.63.65:39387, user_credentials={real_user=slave}} blocked reactor thread for 135302us
I20260806 01:07:45.337328 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 31706
I20260806 01:07:45.400525 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.382768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.22.63.125:35243)
0806 01:07:45.382990 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.383006 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.384165 (+ 1159us) server_negotiation.cc:407] Connection header received
0806 01:07:45.384482 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.384489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.384582 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.384666 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:45.387613 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.388424 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.390268 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.390963 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.391086 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.394215 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:45.394221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:45.394230 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:45.394264 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:45.397592 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.398339 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.398346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.398349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.398484 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.398951 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.398956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.398959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.399309 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:45.399576 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.399950 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.400176 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:07:45.401255 1083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:45.384211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.22.63.124:38357)
0806 01:07:45.384476 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:45.384482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:45.384719 (+ 237us) server_negotiation.cc:407] Connection header received
0806 01:07:45.384882 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:45.384889 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:45.384972 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:45.385074 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:45.387545 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.388716 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.390666 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:45.391193 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:45.391326 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:45.394879 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:45.394888 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:45.394895 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:45.394942 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:45.397716 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.398554 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.398562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.398567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.398708 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:45.399135 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:45.399140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:45.399143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:45.399501 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:45.399760 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:45.400277 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 01:07:45.400509 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:07:46.129273 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.128267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:40947)
0806 01:07:46.128836 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.129028 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
W20260806 01:07:46.129801 865 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:45699 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.134985 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.134196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:47813)
0806 01:07:46.134705 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.134830 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":288,"threads_started":1}
W20260806 01:07:46.135504 673 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:45699 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.138010 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.137213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:35257)
0806 01:07:46.137712 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.137829 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":292,"threads_started":1}
W20260806 01:07:46.138373 1073 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:45699 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.648124 1180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a3b57e3ad26495ebde108ea7f432867)
I20260806 01:07:46.648418 1180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:46.648933 1181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a3b57e3ad26495ebde108ea7f432867)
I20260806 01:07:46.649175 1181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:46.649928 1180 leader_election.cc:288] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124:38357)
I20260806 01:07:46.650527 1181 leader_election.cc:288] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125:35243)
I20260806 01:07:46.651834 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.650129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33558)
0806 01:07:46.651187 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.651436 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":890,"threads_started":1}
W20260806 01:07:46.653187 32599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.661484 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.653762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33560)
0806 01:07:46.660906 (+ 7144us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.661155 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7001,"thread_start_us":6850,"threads_started":1}
W20260806 01:07:46.662060 404 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.663280 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.650411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:38357 (local address 127.0.0.1:60650)
0806 01:07:46.652062 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.652085 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:46.652262 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:46.654163 (+ 1901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:46.654172 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:46.654211 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:46.654546 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:46.654562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.656972 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.656982 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:46.659665 (+ 2683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:46.659686 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.662239 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.662248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:46.662551 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:46.662611 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 01:07:46.662698 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 01:07:46.662849 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1532,"mutex_wait_us":3,"thread_start_us":445,"threads_started":1}
I20260806 01:07:46.663312 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.650812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35243 (local address 127.0.0.1:52294)
0806 01:07:46.653582 (+ 2770us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.653601 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:46.653735 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:46.654322 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:46.654330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:46.654367 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:46.654750 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:46.654765 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.657199 (+ 2434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.657207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:46.660153 (+ 2946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:46.660177 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.662357 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.662363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:46.662571 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:46.662619 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:07:46.662717 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:07:46.662864 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2616,"mutex_wait_us":85,"thread_start_us":3055,"threads_started":1}
I20260806 01:07:46.664040 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.663515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33564)
0806 01:07:46.663720 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.663864 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 01:07:46.664206 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.650401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.22.63.124:38357)
0806 01:07:46.653741 (+ 3340us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:46.653750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:46.653776 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:07:46.653856 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:46.653864 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:46.653959 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:46.654721 (+ 762us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:46.655042 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.656746 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.659965 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.661956 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.663519 (+ 1563us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:46.663622 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:46.663677 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 01:07:46.663835 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":3173,"thread_start_us":3129,"threads_started":1}
I20260806 01:07:46.664355 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.651157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.22.63.125:35243)
0806 01:07:46.653270 (+ 2113us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:46.653278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:46.653853 (+ 575us) server_negotiation.cc:407] Connection header received
0806 01:07:46.653925 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:46.653929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:46.654013 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:46.654134 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:46.655270 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.656944 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.660395 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:46.662081 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:46.662229 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:46.663671 (+ 1442us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:46.663725 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 01:07:46.663906 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1972,"server-negotiator.queue_time_us":1974,"thread_start_us":1371,"threads_started":1}
W20260806 01:07:46.664280 32599 leader_election.cc:334] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.664958 32635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" is_pre_election: true
I20260806 01:07:46.664917 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" is_pre_election: true
I20260806 01:07:46.665292 32635 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d777326282e4cd99dcd7fd6ae52cfae in term 1.
I20260806 01:07:46.665314 437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5086e71fbb3845c0a95b6ab29d26d1c8 in term 1.
I20260806 01:07:46.665820 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.665387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33578)
0806 01:07:46.665537 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.665679 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 01:07:46.665829 408 leader_election.cc:302] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d777326282e4cd99dcd7fd6ae52cfae, 5086e71fbb3845c0a95b6ab29d26d1c8; no voters:
I20260806 01:07:46.666249 1181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 01:07:46.666396 1181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a3b57e3ad26495ebde108ea7f432867)
I20260806 01:07:46.666363 32602 leader_election.cc:302] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d777326282e4cd99dcd7fd6ae52cfae, 5086e71fbb3845c0a95b6ab29d26d1c8; no voters: 3a3b57e3ad26495ebde108ea7f432867
I20260806 01:07:46.666512 1181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:07:46.666831 1180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 01:07:46.666989 1180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a3b57e3ad26495ebde108ea7f432867)
I20260806 01:07:46.667109 1180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 1 FOLLOWER]: Advancing to term 2
W20260806 01:07:46.667083 404 leader_election.cc:334] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.669308 1181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:46.669391 1180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:46.670038 1181 leader_election.cc:288] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 election: Requested vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 5086e71fbb3845c0a95b6ab29d26d1c8 (127.22.63.125:35243)
I20260806 01:07:46.670521 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.670115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33582)
0806 01:07:46.670282 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.670370 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:07:46.670650 32635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8"
I20260806 01:07:46.670727 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
I20260806 01:07:46.670933 32635 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d777326282e4cd99dcd7fd6ae52cfae in current term 2: Already voted for candidate 5086e71fbb3845c0a95b6ab29d26d1c8 in this term.
I20260806 01:07:46.671067 437 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5086e71fbb3845c0a95b6ab29d26d1c8 in current term 2: Already voted for candidate 2d777326282e4cd99dcd7fd6ae52cfae in this term.
I20260806 01:07:46.671352 1180 leader_election.cc:288] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 election: Requested vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124:38357)
I20260806 01:07:46.671711 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.671248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33602)
0806 01:07:46.671393 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.671566 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 01:07:46.671940 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.670449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33586)
0806 01:07:46.671673 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.671801 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1138}
W20260806 01:07:46.671908 404 leader_election.cc:334] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.672044 404 leader_election.cc:302] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d777326282e4cd99dcd7fd6ae52cfae; no voters: 3a3b57e3ad26495ebde108ea7f432867, 5086e71fbb3845c0a95b6ab29d26d1c8
I20260806 01:07:46.672389 1181 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 01:07:46.672910 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:46.672590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:33616)
0806 01:07:46.672721 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:46.672787 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 01:07:46.673125 32599 leader_election.cc:334] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:46.673264 32599 leader_election.cc:302] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5086e71fbb3845c0a95b6ab29d26d1c8; no voters: 2d777326282e4cd99dcd7fd6ae52cfae, 3a3b57e3ad26495ebde108ea7f432867
I20260806 01:07:46.673543 1180 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 01:07:47.131485 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:47.130540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:38569)
0806 01:07:47.131148 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:47.131273 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":352,"threads_started":1}
I20260806 01:07:47.137054 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:47.136264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:55683)
0806 01:07:47.136755 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:47.136839 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":269,"threads_started":1}
I20260806 01:07:47.139669 1198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:47.138952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:33739)
0806 01:07:47.139421 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:47.139499 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":263,"threads_started":1}
I20260806 01:07:48.133395 1200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.132391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:40821)
0806 01:07:48.132996 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.133111 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":318,"threads_started":1}
I20260806 01:07:48.139338 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.138166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:57075)
0806 01:07:48.139021 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.139129 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":399,"threads_started":1}
I20260806 01:07:48.141327 1202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.140591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:39131)
0806 01:07:48.141071 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.141158 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":272,"threads_started":1}
I20260806 01:07:48.559305 1206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:48.559520 1206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:48.560240 1206 leader_election.cc:288] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124:38357)
I20260806 01:07:48.560859 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" is_pre_election: true
I20260806 01:07:48.561131 437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5086e71fbb3845c0a95b6ab29d26d1c8 in term 2.
I20260806 01:07:48.561527 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.560514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42656)
0806 01:07:48.561127 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.561311 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":363,"threads_started":1}
I20260806 01:07:48.561652 32602 leader_election.cc:302] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d777326282e4cd99dcd7fd6ae52cfae, 5086e71fbb3845c0a95b6ab29d26d1c8; no voters:
I20260806 01:07:48.561972 1206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 01:07:48.562141 1206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:48.562260 1206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:07:48.563153 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.562739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42664)
0806 01:07:48.562957 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.563031 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260806 01:07:48.563355 32599 leader_election.cc:334] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:48.564667 1206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:48.565279 1206 leader_election.cc:288] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 election: Requested vote from peers 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699), 2d777326282e4cd99dcd7fd6ae52cfae (127.22.63.124:38357)
I20260806 01:07:48.565847 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.565403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42666)
0806 01:07:48.565569 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.565681 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 01:07:48.565816 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d777326282e4cd99dcd7fd6ae52cfae"
I20260806 01:07:48.566045 437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:07:48.566934 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.566549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42680)
0806 01:07:48.566713 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.566805 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 01:07:48.567135 32599 leader_election.cc:334] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
I20260806 01:07:48.568533 437 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5086e71fbb3845c0a95b6ab29d26d1c8 in term 3.
I20260806 01:07:48.568955 32602 leader_election.cc:302] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d777326282e4cd99dcd7fd6ae52cfae, 5086e71fbb3845c0a95b6ab29d26d1c8; no voters: 3a3b57e3ad26495ebde108ea7f432867
I20260806 01:07:48.569263 1206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:07:48.569803 1206 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [term 3 LEADER]: Becoming Leader. State: Replica: 5086e71fbb3845c0a95b6ab29d26d1c8, State: Running, Role: LEADER
I20260806 01:07:48.570212 1206 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } }
I20260806 01:07:48.572037 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5086e71fbb3845c0a95b6ab29d26d1c8. Latest consensus state: current_term: 3 leader_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:48.572431 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:48.573186 1212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:48.577097 1212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:48.578016 1212 catalog_manager.cc:1259] Loaded cluster ID: 86921016eb8a44149bd72d50d420a6a2
I20260806 01:07:48.578086 1212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:48.579452 1212 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:48.580396 1212 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Loaded TSK: 0
I20260806 01:07:48.581277 1212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:07:48.601833 32618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:48.608916 32618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:48.613300 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.612853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42688)
0806 01:07:48.613078 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.613155 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 01:07:48.613533 437 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [term 3 FOLLOWER]: Refusing update from remote peer 5086e71fbb3845c0a95b6ab29d26d1c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 01:07:48.614462 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.614044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42700)
0806 01:07:48.614268 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.614338 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:07:48.614454 1209 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 01:07:48.614974 32599 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 -> Peer 3a3b57e3ad26495ebde108ea7f432867 (127.22.63.126:45699): Couldn't send request to peer 3a3b57e3ad26495ebde108ea7f432867. Status: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 01:07:48.617758 1215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5086e71fbb3845c0a95b6ab29d26d1c8. Latest consensus state: current_term: 3 leader_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:48.618062 1215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:48.619803 1209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:48.620186 1209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8 [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:48.621133 1215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "3a3b57e3ad26495ebde108ea7f432867" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45699 } } peers { permanent_uuid: "5086e71fbb3845c0a95b6ab29d26d1c8" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35243 } attrs { promote: false } } peers { permanent_uuid: "2d777326282e4cd99dcd7fd6ae52cfae" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 38357 } attrs { promote: false } } }
I20260806 01:07:48.621392 1215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d777326282e4cd99dcd7fd6ae52cfae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:07:48.638314 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.628740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.22.63.65:39387)
0806 01:07:48.630530 (+ 1790us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.630537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.630561 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:07:48.630645 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.630652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.630748 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.630862 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:48.631492 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.632755 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.635854 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.637068 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.637590 (+ 522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.637713 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.637765 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.637908 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1627,"thread_start_us":1497,"threads_started":1}
I20260806 01:07:48.638746 1207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.628566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:39387 (local address 127.0.0.1:46246)
0806 01:07:48.628723 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.628749 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:48.628888 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:48.630907 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:48.630915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:48.630960 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:48.631322 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.631334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.632927 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.632936 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.635629 (+ 2693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.635648 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.637321 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.637330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.637484 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.637588 (+ 104us) client_negotiation.cc:769] Sending connection context
0806 01:07:48.637673 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 01:07:48.638368 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":56}
I20260806 01:07:48.638492 1226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.630276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:46621 (local address 127.0.0.1:51386)
0806 01:07:48.630874 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.630891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:48.630995 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:48.631838 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:48.631844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:48.631872 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:48.632183 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.632194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.633476 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.633482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.635543 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.635555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.637220 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.637233 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.637595 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.637654 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 01:07:48.637763 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 01:07:48.637971 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":115,"thread_start_us":288,"threads_started":1}
I20260806 01:07:48.639058 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.630494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.22.63.66:46621)
0806 01:07:48.631476 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.631481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.631499 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:07:48.631566 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.631571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.631645 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.632358 (+ 713us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:48.632554 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.633340 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.635710 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.637004 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.637167 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.637306 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.638571 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.638727 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":711,"threads_started":1}
I20260806 01:07:48.639761 585 tablet_service.cc:1467] Processing CreateTablet for tablet a387b6960c524b6294ec2b7b1a145639 (DEFAULT_TABLE table=test-table [id=d19ed74180ad4281bc12ac4f0e712c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:48.639892 781 tablet_service.cc:1467] Processing CreateTablet for tablet a387b6960c524b6294ec2b7b1a145639 (DEFAULT_TABLE table=test-table [id=d19ed74180ad4281bc12ac4f0e712c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:48.640429 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a387b6960c524b6294ec2b7b1a145639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:48.640538 585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a387b6960c524b6294ec2b7b1a145639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:48.645563 1228 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Bootstrap starting.
I20260806 01:07:48.647099 1228 tablet_bootstrap.cc:654] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:48.649757 1230 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Bootstrap starting.
I20260806 01:07:48.652195 1230 tablet_bootstrap.cc:654] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:48.653718 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.628885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39683 (local address 127.0.0.1:49118)
0806 01:07:48.629468 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.629487 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:48.629596 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:48.648052 (+ 18456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:48.648058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:48.648088 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:48.648455 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.648470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.650080 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.650086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.651423 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.651436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.652837 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.652843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.653028 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.653054 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:07:48.653174 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 01:07:48.653324 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":365,"threads_started":1}
I20260806 01:07:48.654572 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.629020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.22.63.67:39683)
0806 01:07:48.647551 (+ 18531us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.647558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.647581 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:07:48.647662 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.647667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.647776 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.648633 (+ 857us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:48.648827 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.649931 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.651589 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.652616 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.652770 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.652887 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.654114 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.654260 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18354,"thread_start_us":18828,"threads_started":1}
I20260806 01:07:48.655510 981 tablet_service.cc:1467] Processing CreateTablet for tablet a387b6960c524b6294ec2b7b1a145639 (DEFAULT_TABLE table=test-table [id=d19ed74180ad4281bc12ac4f0e712c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:48.655664 1228 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: No bootstrap required, opened a new log
I20260806 01:07:48.655872 1228 ts_tablet_manager.cc:1397] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260806 01:07:48.656031 981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a387b6960c524b6294ec2b7b1a145639. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:48.656711 1228 raft_consensus.cc:348] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.656952 1228 raft_consensus.cc:374] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:48.657039 1230 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: No bootstrap required, opened a new log
I20260806 01:07:48.657047 1228 raft_consensus.cc:729] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84087abafdfd4311b171b1e151889812, State: Initialized, Role: FOLLOWER
I20260806 01:07:48.657193 1230 ts_tablet_manager.cc:1397] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 01:07:48.657346 1228 consensus_queue.cc:260] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.658010 1230 raft_consensus.cc:348] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.658265 1230 raft_consensus.cc:374] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:48.658375 1230 raft_consensus.cc:729] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 345b5bd3e2d745f399caa6fd83dc1670, State: Initialized, Role: FOLLOWER
I20260806 01:07:48.658679 1230 consensus_queue.cc:260] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.659555 1230 ts_tablet_manager.cc:1428] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 01:07:48.660764 1228 ts_tablet_manager.cc:1428] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260806 01:07:48.667554 867 heartbeater.cc:503] Master 127.22.63.125:35243 was elected leader, sending a full tablet report...
I20260806 01:07:48.670977 1240 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Bootstrap starting.
I20260806 01:07:48.673718 1240 tablet_bootstrap.cc:654] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:48.674546 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.661044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:53555)
0806 01:07:48.674035 (+ 12991us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.674221 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12783,"threads_started":1}
I20260806 01:07:48.675827 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.660227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:47941)
0806 01:07:48.675334 (+ 15107us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.675511 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14942,"thread_start_us":14777,"threads_started":1}
I20260806 01:07:48.675990 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.675497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:37051)
0806 01:07:48.675708 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.675835 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 01:07:48.676277 1240 tablet_bootstrap.cc:492] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: No bootstrap required, opened a new log
I20260806 01:07:48.676443 1240 ts_tablet_manager.cc:1397] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 01:07:48.676829 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.676487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:33385)
0806 01:07:48.676650 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.676725 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:07:48.677281 1240 raft_consensus.cc:348] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.677510 1240 raft_consensus.cc:374] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:48.677611 1240 raft_consensus.cc:729] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8cbbbcdfe5d450fb40a1dbcaf1d7f28, State: Initialized, Role: FOLLOWER
I20260806 01:07:48.677852 1240 consensus_queue.cc:260] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.678735 1240 ts_tablet_manager.cc:1428] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:07:48.680351 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.679202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:54691)
0806 01:07:48.680055 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.680170 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":562,"threads_started":1}
I20260806 01:07:48.681555 1077 heartbeater.cc:503] Master 127.22.63.125:35243 was elected leader, sending a full tablet report...
I20260806 01:07:48.681878 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.681490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:34465)
0806 01:07:48.681652 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.681733 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 01:07:48.702039 868 tablet.cc:2369] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:48.712311 1235 raft_consensus.cc:482] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:48.712604 1235 raft_consensus.cc:504] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.714258 1235 leader_election.cc:288] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683), 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:48.721091 1234 raft_consensus.cc:482] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:48.721496 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.714638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:37553 (local address 127.22.63.67:39683)
0806 01:07:48.714873 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.714882 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.714926 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:07:48.715090 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.715094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.715157 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.715261 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:48.715900 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.716974 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.719330 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.720335 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.720472 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.720527 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.721066 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.721219 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:07:48.721328 1234 raft_consensus.cc:504] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.721575 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.714444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39683 (local address 127.22.63.65:37553)
0806 01:07:48.714723 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.714744 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:48.714900 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:48.715292 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:48.715297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:48.715325 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:48.715668 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.715681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.717121 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.717128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.719141 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.719155 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.720552 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.720559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.720777 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.720822 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 01:07:48.721023 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 01:07:48.721206 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":47}
I20260806 01:07:48.721720 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.714587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:46621 (local address 127.22.63.65:58065)
0806 01:07:48.715410 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.715428 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:48.715536 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:48.715914 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:48.715919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:48.715946 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:48.716271 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.716283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.717699 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.717711 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.719435 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:48.719453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.720826 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.720830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:48.721093 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.721126 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:07:48.721212 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 01:07:48.721329 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":1,"thread_start_us":471,"threads_started":1}
I20260806 01:07:48.722277 1234 leader_election.cc:288] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387), c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:48.722276 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "345b5bd3e2d745f399caa6fd83dc1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" is_pre_election: true
I20260806 01:07:48.722545 1007 raft_consensus.cc:2454] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345b5bd3e2d745f399caa6fd83dc1670 in term 0.
I20260806 01:07:48.722754 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.714643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:58065 (local address 127.22.63.66:46621)
0806 01:07:48.714884 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.714890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.715525 (+ 635us) server_negotiation.cc:407] Connection header received
0806 01:07:48.715658 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.715663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.715787 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.715885 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:48.716508 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.717521 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.719603 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.720604 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.720787 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.720871 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.722122 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.722253 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:07:48.723011 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" is_pre_election: true
I20260806 01:07:48.723079 614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345b5bd3e2d745f399caa6fd83dc1670" is_pre_election: true
I20260806 01:07:48.723220 1007 raft_consensus.cc:2454] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 0.
I20260806 01:07:48.723169 515 leader_election.cc:302] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 345b5bd3e2d745f399caa6fd83dc1670, c8cbbbcdfe5d450fb40a1dbcaf1d7f28; no voters:
I20260806 01:07:48.723371 614 raft_consensus.cc:2454] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84087abafdfd4311b171b1e151889812 in term 0.
I20260806 01:07:48.723594 711 leader_election.cc:302] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84087abafdfd4311b171b1e151889812, c8cbbbcdfe5d450fb40a1dbcaf1d7f28; no voters:
I20260806 01:07:48.723775 1235 raft_consensus.cc:2781] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:48.723915 1235 raft_consensus.cc:482] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:48.724017 1235 raft_consensus.cc:3037] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:48.723942 809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "345b5bd3e2d745f399caa6fd83dc1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84087abafdfd4311b171b1e151889812" is_pre_election: true
I20260806 01:07:48.724156 1234 raft_consensus.cc:2781] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:48.724429 809 raft_consensus.cc:2454] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 345b5bd3e2d745f399caa6fd83dc1670 in term 0.
I20260806 01:07:48.724686 1234 raft_consensus.cc:482] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:48.724824 1234 raft_consensus.cc:3037] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:48.727636 1235 raft_consensus.cc:504] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.728140 1234 raft_consensus.cc:504] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.728588 1235 leader_election.cc:288] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [CANDIDATE]: Term 1 election: Requested vote from peers c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683), 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621)
I20260806 01:07:48.728865 1234 leader_election.cc:288] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 election: Requested vote from peers 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387), c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683)
I20260806 01:07:48.729156 1007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "345b5bd3e2d745f399caa6fd83dc1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
I20260806 01:07:48.729283 614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "345b5bd3e2d745f399caa6fd83dc1670"
I20260806 01:07:48.729382 1007 raft_consensus.cc:3037] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:48.729352 809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "345b5bd3e2d745f399caa6fd83dc1670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84087abafdfd4311b171b1e151889812"
I20260806 01:07:48.729570 614 raft_consensus.cc:2379] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 84087abafdfd4311b171b1e151889812 in current term 1: Already voted for candidate 345b5bd3e2d745f399caa6fd83dc1670 in this term.
I20260806 01:07:48.729624 809 raft_consensus.cc:2379] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 345b5bd3e2d745f399caa6fd83dc1670 in current term 1: Already voted for candidate 84087abafdfd4311b171b1e151889812 in this term.
I20260806 01:07:48.730216 1006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a387b6960c524b6294ec2b7b1a145639" candidate_uuid: "84087abafdfd4311b171b1e151889812" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28"
I20260806 01:07:48.730829 711 leader_election.cc:302] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84087abafdfd4311b171b1e151889812; no voters: 345b5bd3e2d745f399caa6fd83dc1670, c8cbbbcdfe5d450fb40a1dbcaf1d7f28
I20260806 01:07:48.731161 1234 raft_consensus.cc:2726] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 01:07:48.732486 1007 raft_consensus.cc:2454] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 345b5bd3e2d745f399caa6fd83dc1670 in term 1.
I20260806 01:07:48.732815 515 leader_election.cc:302] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 345b5bd3e2d745f399caa6fd83dc1670, c8cbbbcdfe5d450fb40a1dbcaf1d7f28; no voters: 84087abafdfd4311b171b1e151889812
I20260806 01:07:48.733192 1235 raft_consensus.cc:2781] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:48.733765 1235 raft_consensus.cc:686] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 LEADER]: Becoming Leader. State: Replica: 345b5bd3e2d745f399caa6fd83dc1670, State: Running, Role: LEADER
I20260806 01:07:48.734261 675 heartbeater.cc:503] Master 127.22.63.125:35243 was elected leader, sending a full tablet report...
I20260806 01:07:48.734331 1235 consensus_queue.cc:237] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } }
I20260806 01:07:48.735215 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.734733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:56001)
0806 01:07:48.734943 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.735046 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:07:48.738118 32618 catalog_manager.cc:5495] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 reported cstate change: term changed from 0 to 1, leader changed from <none> to 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "345b5bd3e2d745f399caa6fd83dc1670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "345b5bd3e2d745f399caa6fd83dc1670" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 39387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:48.769573 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.763322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.22.63.65:39387)
0806 01:07:48.763586 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.763593 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.763619 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:07:48.763801 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.763807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.763923 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.764099 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:48.764509 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.765499 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.767070 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.767534 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.767702 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.768202 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:48.768891 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:48.769061 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.769152 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.769247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:07:48.774288 1007 raft_consensus.cc:1261] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Refusing update from remote peer 345b5bd3e2d745f399caa6fd83dc1670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:48.774538 809 raft_consensus.cc:1261] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Refusing update from remote peer 345b5bd3e2d745f399caa6fd83dc1670: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:48.775049 1246 consensus_queue.cc:1035] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8cbbbcdfe5d450fb40a1dbcaf1d7f28" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:48.775413 1235 consensus_queue.cc:1035] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84087abafdfd4311b171b1e151889812" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:48.778890 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.778170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:58689)
0806 01:07:48.778457 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.778563 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260806 01:07:48.779016 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.778442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:44563)
0806 01:07:48.778645 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.778793 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 01:07:48.784143 1250 mvcc.cc:205] Tried to move back new op lower bound from 7315367808091496448 to 7315367807940268032. Current Snapshot: MvccSnapshot[applied={T|T < 7315367808091496448}]
I20260806 01:07:48.785630 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.785030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:47429)
0806 01:07:48.785273 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.785414 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 01:07:48.786850 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.786217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.66:55245)
0806 01:07:48.786516 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.786636 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 01:07:48.788630 1253 mvcc.cc:205] Tried to move back new op lower bound from 7315367808091496448 to 7315367807940268032. Current Snapshot: MvccSnapshot[applied={T|T < 7315367808091496448}]
I20260806 01:07:48.799472 32618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:48.799926 32618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:48.803583 1244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.802633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.65:44219)
0806 01:07:48.803250 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.803372 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472}
I20260806 01:07:48.808493 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.807814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.22.63.67:51647)
0806 01:07:48.808184 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:48.808308 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256}
I20260806 01:07:48.812328 32618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5086e71fbb3845c0a95b6ab29d26d1c8: Sending DeleteTablet for 3 replicas of tablet a387b6960c524b6294ec2b7b1a145639
I20260806 01:07:48.813429 585 tablet_service.cc:1514] Processing DeleteTablet for tablet a387b6960c524b6294ec2b7b1a145639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:48 UTC) from {username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:46246
I20260806 01:07:48.813951 981 tablet_service.cc:1514] Processing DeleteTablet for tablet a387b6960c524b6294ec2b7b1a145639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:48 UTC) from {username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:49118
I20260806 01:07:48.814707 781 tablet_service.cc:1514] Processing DeleteTablet for tablet a387b6960c524b6294ec2b7b1a145639 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:48 UTC) from {username='oryx', principal='oryx/127.22.63.125@KRBTEST.COM'} at 127.0.0.1:51386
I20260806 01:07:48.820854 1268 tablet_replica.cc:330] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: stopping tablet replica
I20260806 01:07:48.821035 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 388
I20260806 01:07:48.821105 1268 raft_consensus.cc:2229] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:48.821455 1268 raft_consensus.cc:2258] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:48.822117 1267 tablet_replica.cc:330] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: stopping tablet replica
I20260806 01:07:48.822739 1266 tablet_replica.cc:330] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: stopping tablet replica
I20260806 01:07:48.823017 1266 raft_consensus.cc:2229] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:48.823726 1266 raft_consensus.cc:2258] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:48.824522 1268 ts_tablet_manager.cc:1905] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:48.831274 1268 ts_tablet_manager.cc:1918] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:48.831419 1268 log.cc:1196] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/a387b6960c524b6294ec2b7b1a145639
I20260806 01:07:48.831888 1268 ts_tablet_manager.cc:1939] T a387b6960c524b6294ec2b7b1a145639 P 84087abafdfd4311b171b1e151889812: Deleting consensus metadata
I20260806 01:07:48.832661 1266 ts_tablet_manager.cc:1905] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:48.833209 32598 catalog_manager.cc:4841] TS 84087abafdfd4311b171b1e151889812 (127.22.63.66:46621): tablet a387b6960c524b6294ec2b7b1a145639 (table test-table [id=d19ed74180ad4281bc12ac4f0e712c41]) successfully deleted
I20260806 01:07:48.822314 1267 raft_consensus.cc:2229] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:48.833782 1267 raft_consensus.cc:2258] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:48.834995 1267 ts_tablet_manager.cc:1905] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:48.841960 1267 ts_tablet_manager.cc:1918] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:48.842115 1267 log.cc:1196] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/a387b6960c524b6294ec2b7b1a145639
I20260806 01:07:48.842566 1267 ts_tablet_manager.cc:1939] T a387b6960c524b6294ec2b7b1a145639 P c8cbbbcdfe5d450fb40a1dbcaf1d7f28: Deleting consensus metadata
I20260806 01:07:48.843713 32599 catalog_manager.cc:4841] TS c8cbbbcdfe5d450fb40a1dbcaf1d7f28 (127.22.63.67:39683): tablet a387b6960c524b6294ec2b7b1a145639 (table test-table [id=d19ed74180ad4281bc12ac4f0e712c41]) successfully deleted
I20260806 01:07:48.846685 1266 ts_tablet_manager.cc:1918] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:48.846892 1266 log.cc:1196] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/a387b6960c524b6294ec2b7b1a145639
I20260806 01:07:48.847599 1266 ts_tablet_manager.cc:1939] T a387b6960c524b6294ec2b7b1a145639 P 345b5bd3e2d745f399caa6fd83dc1670: Deleting consensus metadata
I20260806 01:07:48.848776 32602 catalog_manager.cc:4841] TS 345b5bd3e2d745f399caa6fd83dc1670 (127.22.63.65:39387): tablet a387b6960c524b6294ec2b7b1a145639 (table test-table [id=d19ed74180ad4281bc12ac4f0e712c41]) successfully deleted
I20260806 01:07:48.858913 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.821996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.22.63.65:39387)
0806 01:07:48.822227 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.822238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.840413 (+ 18175us) server_negotiation.cc:407] Connection header received
0806 01:07:48.840654 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.840660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.840776 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.840895 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:48.843367 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.844776 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.846929 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.847913 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.848027 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.852369 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:48.852385 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:48.852402 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:48.852458 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:48.855908 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:48.856651 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:48.856660 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:48.856665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:48.856817 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:48.857236 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:48.857243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:48.857246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:48.857613 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:48.857961 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:48.858299 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 01:07:48.858557 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:07:48.859845 666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:48.965075 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 388
I20260806 01:07:48.991550 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 685
I20260806 01:07:49.005625 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:48.992416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51402 (local address 127.22.63.66:46621)
0806 01:07:48.992632 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:48.992639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:48.992747 (+ 108us) server_negotiation.cc:407] Connection header received
0806 01:07:48.993014 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:48.993019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:48.993098 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:48.993212 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:48.994572 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.995289 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.996709 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:48.997118 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:48.997265 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:48.999780 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:48.999805 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:48.999823 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:48.999867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:49.002630 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:49.003368 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:49.003380 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:49.003382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:49.003506 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:49.003979 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:49.003985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:49.003988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:49.004394 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:49.004740 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:49.005022 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 01:07:49.005244 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:07:49.006296 858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:49.076730 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:49.076262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42708)
0806 01:07:49.076488 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:49.076584 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:07:49.077752 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:49.077388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45699 (local address 127.0.0.1:42716)
0806 01:07:49.077533 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:49.077621 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:45699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 01:07:49.128553 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 685
I20260806 01:07:49.162742 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 887
I20260806 01:07:49.177385 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:49.163543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.22.63.67:39683)
0806 01:07:49.163726 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:49.163731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:49.163801 (+ 70us) server_negotiation.cc:407] Connection header received
0806 01:07:49.163996 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:49.164000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:49.164070 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:49.164158 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:49.165773 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:49.166547 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:49.167703 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:49.168273 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:49.168372 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:49.171235 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:49.171251 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:49.171268 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:49.171304 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:49.174370 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:49.175090 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:49.175099 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:49.175101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:49.175258 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:49.175640 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:49.175644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:49.175646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:49.176262 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:49.176511 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:49.176778 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:07:49.176989 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:07:49.178087 1063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:49.276467 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 887
I20260806 01:07:49.300904 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 32425
I20260806 01:07:49.301986 32665 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:49.428063 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 32425
W20260806 01:07:49.457556 404 connection.cc:535] server connection from 127.0.0.1:60650 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:07:49.457944 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 32557
I20260806 01:07:49.459064 466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:49.577373 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 32557
2026-08-06T01:07:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11923 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:49 dist-test-slave-7f01 krb5kdc[1280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:49 dist-test-slave-7f01 krb5kdc[1280](info): set up 2 sockets
Aug 06 01:07:49 dist-test-slave-7f01 krb5kdc[1280](info): commencing operation
krb5kdc: starting...
W20260806 01:07:50.918574 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:51 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab.
Entry for principal oryx/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab.
I20260806 01:07:51.189612 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40555
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab
--principal=oryx/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:51.535115 1304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:51.535491 1304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:51.535668 1304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:51.543000 1304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:51.543162 1304 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 01:07:51.543262 1304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:51.543381 1304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:51.543471 1304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:51.543563 1304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:51.553979 1304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40555
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.126.keytab
--principal=oryx/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:51.555431 1304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:51.556900 1304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:51.569450 1312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:51.569510 1309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:51.569499 1310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:51.570616 1304 server_base.cc:1034] running on GCE node
I20260806 01:07:51.571439 1304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:51.572994 1304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:51.574214 1304 hybrid_clock.cc:648] HybridClock initialized: now 1785978471574155 us; error 46 us; skew 500 ppm
I20260806 01:07:51.574522 1304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:51 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978471, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:51.577828 1304 init.cc:377] Logged in from keytab as oryx/127.22.63.126@KRBTEST.COM (short username oryx)
I20260806 01:07:51.580123 1304 webserver.cc:460] Webserver started at http://127.22.63.126:33545/ using document root <none> and password file <none>
I20260806 01:07:51.580871 1304 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:51.580993 1304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:51.581328 1304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:51.584429 1304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "44b0da2b8d7245a68b62569269d04b1f"
format_stamp: "Formatted at 2026-08-06 01:07:51 on dist-test-slave-7f01"
server_key: "bafdaea16059900bcb9f55a9668adb1b"
server_key_iv: "7ad6a66020f3b738f67885626a236bd7"
server_key_version: "encryptionkey@0"
I20260806 01:07:51.585265 1304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "44b0da2b8d7245a68b62569269d04b1f"
format_stamp: "Formatted at 2026-08-06 01:07:51 on dist-test-slave-7f01"
server_key: "bafdaea16059900bcb9f55a9668adb1b"
server_key_iv: "7ad6a66020f3b738f67885626a236bd7"
server_key_version: "encryptionkey@0"
I20260806 01:07:51.590572 1304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:07:51.594072 1330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:51.595423 1304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:51.595603 1304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "44b0da2b8d7245a68b62569269d04b1f"
format_stamp: "Formatted at 2026-08-06 01:07:51 on dist-test-slave-7f01"
server_key: "bafdaea16059900bcb9f55a9668adb1b"
server_key_iv: "7ad6a66020f3b738f67885626a236bd7"
server_key_version: "encryptionkey@0"
I20260806 01:07:51.595893 1304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:51.611397 1304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:51.615537 1304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:51.615877 1304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:51.638441 1304 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40555
I20260806 01:07:51.638463 1423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40555 every 8 connection(s)
I20260806 01:07:51.639956 1304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:07:51.643888 1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:51.647982 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 1304
I20260806 01:07:51.648214 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:07:51.648886 22781 external_mini_cluster.cc:1428] Setting key 90d7848b4a73ba21e1b57f834ca0f131
I20260806 01:07:51.655422 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Bootstrap starting.
I20260806 01:07:51.659211 1424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:51.660490 1424 log.cc:824] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:07:51 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:51.664043 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: No bootstrap required, opened a new log
I20260806 01:07:51.673650 1428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:51.650939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.22.63.126:40555)
0806 01:07:51.651820 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:51.651832 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:51.651879 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:07:51.653775 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:51.653799 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:51.654208 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:51.654527 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:51.656416 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:51.657735 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:51.659514 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:51.660280 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:51.660422 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:51.665065 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:51.665076 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:51.665095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:51.665143 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:51.669283 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:51.670018 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:51.670028 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:51.670042 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:51.670270 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:51.670684 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:51.670688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:51.670690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:51.671284 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:51.671632 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:51.672115 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 01:07:51.672573 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":382,"threads_started":1}
I20260806 01:07:51.674355 1424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } }
I20260806 01:07:51.674738 1424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:51.674881 1424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b0da2b8d7245a68b62569269d04b1f, State: Initialized, Role: FOLLOWER
I20260806 01:07:51.675455 1424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } }
I20260806 01:07:51.675717 1424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:07:51.675926 1424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:07:51.676127 1424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:51.678505 1424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } }
I20260806 01:07:51.679041 1424 leader_election.cc:302] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44b0da2b8d7245a68b62569269d04b1f; no voters:
I20260806 01:07:51.679644 1424 leader_election.cc:288] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:07:51.679828 1432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:51.680642 1432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [term 1 LEADER]: Becoming Leader. State: Replica: 44b0da2b8d7245a68b62569269d04b1f, State: Running, Role: LEADER
I20260806 01:07:51.681150 1432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } }
I20260806 01:07:51.682176 1424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:07:51.684213 1434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44b0da2b8d7245a68b62569269d04b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } } }
I20260806 01:07:51.684154 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44b0da2b8d7245a68b62569269d04b1f. Latest consensus state: current_term: 1 leader_uuid: "44b0da2b8d7245a68b62569269d04b1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44b0da2b8d7245a68b62569269d04b1f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40555 } } }
I20260806 01:07:51.684610 1434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:51.684614 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f [sys.catalog]: This master's current role is: LEADER
I20260806 01:07:51.686432 1440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:07:51.691193 1440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:07:51.699266 1440 catalog_manager.cc:1347] Generated new cluster ID: c6f6549afe7445929bbd25952f57728d
I20260806 01:07:51.699429 1440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:07:51.741272 1440 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:07:51.743182 1440 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:07:51.760843 1440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Generated new TSK 0
I20260806 01:07:51.761910 1440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
I20260806 01:07:51.836037 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40555
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:52.193965 1463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:52.194249 1463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:52.194439 1463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:52.201884 1463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:52.202004 1463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:52.202180 1463 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:07:52.202309 1463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:07:52.212188 1463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:52.213538 1463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:52.214959 1463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:52.229185 1476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:52.229202 1472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:52.229151 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:52.230206 1463 server_base.cc:1034] running on GCE node
I20260806 01:07:52.230834 1463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:52.231853 1463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:52.233096 1463 hybrid_clock.cc:648] HybridClock initialized: now 1785978472233037 us; error 32 us; skew 500 ppm
I20260806 01:07:52.233376 1463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:52 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978472, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:52.236738 1463 init.cc:377] Logged in from keytab as oryx/127.22.63.65@KRBTEST.COM (short username oryx)
I20260806 01:07:52.238235 1463 webserver.cc:460] Webserver started at http://127.22.63.65:46565/ using document root <none> and password file <none>
I20260806 01:07:52.238984 1463 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:52.239076 1463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:52.239445 1463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:52.242528 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "7bc7c70276af41079fa30e7b455ca47e"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "f9942e19770b2872538b30657c977bb5"
server_key_iv: "5185cd333b2eebed0c7a528a3364d6e3"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.243363 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "7bc7c70276af41079fa30e7b455ca47e"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "f9942e19770b2872538b30657c977bb5"
server_key_iv: "5185cd333b2eebed0c7a528a3364d6e3"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.248772 1463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:07:52.252313 1485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:52.253608 1463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:07:52.253784 1463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "7bc7c70276af41079fa30e7b455ca47e"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "f9942e19770b2872538b30657c977bb5"
server_key_iv: "5185cd333b2eebed0c7a528a3364d6e3"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.254014 1463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:52.273048 1463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:52.277714 1463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:52.278039 1463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:52.281041 1463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:52.281157 1463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:52.281322 1463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:52.281407 1463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:07:52 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978472, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:52.307286 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.281597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54677 (local address 127.22.63.126:40555)
0806 01:07:52.282397 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:52.282404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:52.283474 (+ 1070us) server_negotiation.cc:407] Connection header received
0806 01:07:52.285626 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:52.285631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:52.285728 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:52.285859 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:52.288236 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.289335 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.292265 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.292708 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.292869 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.299086 (+ 6217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:52.299095 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:52.299101 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:52.299149 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:52.302455 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.303697 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.303702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.303707 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.303907 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.304620 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.304629 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.304635 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.305090 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:52.305288 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:52.306743 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0806 01:07:52.307011 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":522,"threads_started":1}
I20260806 01:07:52.309588 1499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.282327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.65:54677)
0806 01:07:52.283266 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:52.283308 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:52.285322 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:52.286116 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:52.286124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:52.286943 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:52.288028 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.288069 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.289521 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.289530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.292078 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.292091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.293022 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.293033 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.293314 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.294385 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:52.294430 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:52.298748 (+ 4318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:52.302634 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.302650 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.302673 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.303511 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.304053 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.304060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.304067 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.304432 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.305489 (+ 1057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:52.305499 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:52.306181 (+ 682us) client_negotiation.cc:769] Sending connection context
0806 01:07:52.306571 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 01:07:52.307385 (+ 814us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":391,"threads_started":1}
I20260806 01:07:52.334570 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.318634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:50723 (local address 127.22.63.126:40555)
0806 01:07:52.318800 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:52.318805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:52.319833 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 01:07:52.320192 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:52.320198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:52.320323 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:52.320411 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:52.322241 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.322979 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.324966 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.325401 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.325547 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.327536 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:52.327540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:52.327544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:52.327576 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:52.330929 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.332149 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.332153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.332155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.332282 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.332773 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.332776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.332778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.333112 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:52.333226 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:52.333840 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 01:07:52.334115 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:07:52.334883 1616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.318750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.65:50723)
0806 01:07:52.319686 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:52.319711 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:52.319998 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:52.320661 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:52.320673 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:52.321481 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:52.321994 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.322021 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.323136 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.323141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.324807 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.324821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.325600 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.325606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.325777 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.326509 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:52.326527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:52.327337 (+ 810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:52.331104 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.331112 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.331119 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.331997 (+ 878us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.332424 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.332431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.332438 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.332645 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.333242 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:52.333248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:52.333525 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 01:07:52.333696 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 01:07:52.334252 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":522,"threads_started":1}
I20260806 01:07:52.335286 1463 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:42469
I20260806 01:07:52.335306 1662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:42469 every 8 connection(s)
I20260806 01:07:52.336920 1463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:07:52.339341 1663 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40555
I20260806 01:07:52.339610 1663 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:52.340626 1663 heartbeater.cc:511] Master 127.22.63.126:40555 requested a full tablet report, sending...
I20260806 01:07:52.342653 1359 ts_manager.cc:194] Registered new tserver with Master: 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469)
I20260806 01:07:52.344260 1359 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:54677
I20260806 01:07:52.346560 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 1463
I20260806 01:07:52.346788 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:07:52.347467 22781 external_mini_cluster.cc:1428] Setting key d3be04335d21025879a11a4f56bd519f
WARNING: no policy specified for oryx/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
I20260806 01:07:52.409111 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40555
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:52.759366 1667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:52.759665 1667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:52.759837 1667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:52.767365 1667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:52.767481 1667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:52.767568 1667 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:07:52.767653 1667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:07:52.777681 1667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:52.778965 1667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:52.780375 1667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:52.794616 1675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:52.794643 1678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:52.794541 1676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:52.795396 1667 server_base.cc:1034] running on GCE node
I20260806 01:07:52.796170 1667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:52.797165 1667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:52.798441 1667 hybrid_clock.cc:648] HybridClock initialized: now 1785978472798381 us; error 39 us; skew 500 ppm
I20260806 01:07:52.798822 1667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:52 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978472, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:52.802148 1667 init.cc:377] Logged in from keytab as oryx/127.22.63.66@KRBTEST.COM (short username oryx)
I20260806 01:07:52.803711 1667 webserver.cc:460] Webserver started at http://127.22.63.66:45267/ using document root <none> and password file <none>
I20260806 01:07:52.804895 1667 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:52.805054 1667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:52.805430 1667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:52.808641 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "2d1121839e62429ba2354609ac5dde20"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "c6a4b13a4dab25c445c26e6d0a71cbbd"
server_key_iv: "cc43658885bb2b46871528e454a1c27a"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.809499 1667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "2d1121839e62429ba2354609ac5dde20"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "c6a4b13a4dab25c445c26e6d0a71cbbd"
server_key_iv: "cc43658885bb2b46871528e454a1c27a"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.815418 1667 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 01:07:52.819257 1687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:52.820653 1667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:52.820847 1667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "2d1121839e62429ba2354609ac5dde20"
format_stamp: "Formatted at 2026-08-06 01:07:52 on dist-test-slave-7f01"
server_key: "c6a4b13a4dab25c445c26e6d0a71cbbd"
server_key_iv: "cc43658885bb2b46871528e454a1c27a"
server_key_version: "encryptionkey@0"
I20260806 01:07:52.821089 1667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:52.840910 1667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:52.845012 1667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:52.845295 1667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:52.848181 1667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:52.848294 1667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:52.848464 1667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:52.848567 1667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:07:52 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978472, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:52.874014 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.848837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:38697 (local address 127.22.63.126:40555)
0806 01:07:52.849517 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:52.849524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:52.850638 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 01:07:52.853789 (+ 3151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:52.853798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:52.853909 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:52.854053 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:52.856879 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.857798 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.860728 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.861174 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.861348 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.866624 (+ 5276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:52.866630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:52.866633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:52.866668 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:52.869757 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.870828 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.870833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.870835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.870963 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.871444 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.871448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.871450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.871719 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:52.871941 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:52.873036 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0806 01:07:52.873379 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":411,"threads_started":1}
I20260806 01:07:52.874815 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.849491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.66:38697)
0806 01:07:52.850429 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:52.850488 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:52.853402 (+ 2914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:52.854387 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:52.854399 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:52.855408 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:52.856608 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.856655 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.857990 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.857995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.860563 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.860579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.861705 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.861711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.862065 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.862956 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:52.863000 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:52.866354 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:52.869937 (+ 3583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.869950 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.869975 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.870688 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.871081 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.871087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.871091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.871300 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.871912 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:52.871923 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:52.872487 (+ 564us) client_negotiation.cc:769] Sending connection context
0806 01:07:52.872875 (+ 388us) client_negotiation.cc:240] Negotiation successful
0806 01:07:52.873403 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":382,"threads_started":1}
I20260806 01:07:52.898082 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.883063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:34433 (local address 127.22.63.126:40555)
0806 01:07:52.883320 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:52.883329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:52.883991 (+ 662us) server_negotiation.cc:407] Connection header received
0806 01:07:52.884258 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:52.884267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:52.884367 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:52.884507 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:52.885448 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.886829 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.889312 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.889720 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.889870 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.892460 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:52.892465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:52.892469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:52.892502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:52.895187 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.895952 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.895956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.895958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.896086 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:52.896482 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:52.896486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:52.896488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:52.896821 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:52.897001 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:52.897449 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 01:07:52.897708 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:07:52.898181 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:52.883078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.66:34433)
0806 01:07:52.883878 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:52.883895 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:52.884074 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:52.884529 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:52.884533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:52.884992 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:52.885284 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.885293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.887035 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.887046 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.889069 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:52.889108 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:52.889945 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:52.889954 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:52.890123 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:52.890874 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:52.890898 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:52.892245 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:52.895325 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.895332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.895336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.895792 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.896207 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:52.896212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:52.896216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:52.896349 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:52.896969 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:52.896974 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:52.897172 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 01:07:52.897324 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 01:07:52.897683 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":376,"threads_started":1}
I20260806 01:07:52.907295 1667 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:34283
I20260806 01:07:52.907325 1860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:34283 every 8 connection(s)
I20260806 01:07:52.908663 1667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:07:52.910681 1862 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40555
I20260806 01:07:52.910895 1862 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:52.911512 1862 heartbeater.cc:511] Master 127.22.63.126:40555 requested a full tablet report, sending...
I20260806 01:07:52.913074 1359 ts_manager.cc:194] Registered new tserver with Master: 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283)
I20260806 01:07:52.913954 1359 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:38697
I20260806 01:07:52.918511 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 1667
I20260806 01:07:52.918699 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:07:52.919162 22781 external_mini_cluster.cc:1428] Setting key ec8e9b1067810fee6fe84447205be197
WARNING: no policy specified for oryx/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
I20260806 01:07:52.982888 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40555
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:39715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:53.330718 1868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:53.331036 1868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:53.331139 1868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:53.338649 1868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:53.338814 1868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:53.338970 1868 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 01:07:53.339116 1868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:07:53.347071 1663 heartbeater.cc:503] Master 127.22.63.126:40555 was elected leader, sending a full tablet report...
I20260806 01:07:53.349149 1868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.63.126:40555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:53.350455 1868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:53.351874 1868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:53.365473 1875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:53.365561 1877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:53.365537 1880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:53.366357 1868 server_base.cc:1034] running on GCE node
I20260806 01:07:53.367166 1868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:53.368191 1868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:53.369709 1868 hybrid_clock.cc:648] HybridClock initialized: now 1785978473369516 us; error 58 us; skew 500 ppm
I20260806 01:07:53.370420 1868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:07:53 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978473, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:07:53.373960 1868 init.cc:377] Logged in from keytab as oryx/127.22.63.67@KRBTEST.COM (short username oryx)
I20260806 01:07:53.375487 1868 webserver.cc:460] Webserver started at http://127.22.63.67:36645/ using document root <none> and password file <none>
I20260806 01:07:53.376293 1868 fs_manager.cc:359] Metadata directory not provided
I20260806 01:07:53.376389 1868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:07:53.376758 1868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:07:53.380157 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
format_stamp: "Formatted at 2026-08-06 01:07:53 on dist-test-slave-7f01"
server_key: "88394b5d459da4c44356187b802ddf56"
server_key_iv: "35b5d7edf4d319199387bba07cd907ff"
server_key_version: "encryptionkey@0"
I20260806 01:07:53.381037 1868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
format_stamp: "Formatted at 2026-08-06 01:07:53 on dist-test-slave-7f01"
server_key: "88394b5d459da4c44356187b802ddf56"
server_key_iv: "35b5d7edf4d319199387bba07cd907ff"
server_key_version: "encryptionkey@0"
I20260806 01:07:53.386824 1868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:07:53.390758 1889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:07:53.392316 1868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:53.392503 1868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
format_stamp: "Formatted at 2026-08-06 01:07:53 on dist-test-slave-7f01"
server_key: "88394b5d459da4c44356187b802ddf56"
server_key_iv: "35b5d7edf4d319199387bba07cd907ff"
server_key_version: "encryptionkey@0"
I20260806 01:07:53.392750 1868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:07:53.423230 1868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:07:53.428122 1868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:07:53.428423 1868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:07:53.431669 1868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:07:53.431797 1868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:07:53.431990 1868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:07:53.432070 1868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 01:07:53 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978473, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:53.458745 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:53.432360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:58533 (local address 127.22.63.126:40555)
0806 01:07:53.435273 (+ 2913us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:53.435277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:53.435298 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:07:53.437733 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:53.437741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:53.437847 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:53.437986 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:53.440501 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.441605 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.444016 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.444630 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.444769 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:53.450300 (+ 5531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:53.450307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:53.450312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:53.450366 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:53.454375 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:53.455356 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:53.455365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:53.455371 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:53.455595 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:53.456140 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:53.456147 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:53.456153 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:53.456646 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:53.456842 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:53.457783 (+ 941us) server_negotiation.cc:299] Negotiation successful
0806 01:07:53.458100 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2802,"thread_start_us":2692,"threads_started":1}
I20260806 01:07:53.459723 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:53.433175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.67:58533)
0806 01:07:53.434263 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:53.434321 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:53.437339 (+ 3018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:53.438339 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:53.438354 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:53.439240 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:53.440224 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:53.440258 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.441856 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.441862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:53.443781 (+ 1919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:53.443799 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.444965 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.444972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:53.445163 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:53.446088 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:53.446112 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:53.449974 (+ 3862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:53.454526 (+ 4552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:53.454543 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:53.454575 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:53.455173 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:53.455773 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:53.455781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:53.455784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:53.456009 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:53.456859 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:53.456866 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:53.457304 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 01:07:53.457608 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 01:07:53.458039 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":456,"threads_started":1}
I20260806 01:07:53.485860 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:53.466268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:40245 (local address 127.22.63.126:40555)
0806 01:07:53.466480 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:53.466488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:53.468580 (+ 2092us) server_negotiation.cc:407] Connection header received
0806 01:07:53.468929 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:53.468938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:53.469048 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:53.469126 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:53.471292 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.472232 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.474641 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.475062 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.475171 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:53.478121 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:53.478127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:53.478131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:53.478169 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:53.481933 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:53.483198 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:53.483203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:53.483205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:53.483398 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:53.484071 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:53.484078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:53.484085 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:53.484485 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:53.484672 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:53.485238 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 01:07:53.485494 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:07:53.485890 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:53.466333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.22.63.67:40245)
0806 01:07:53.468444 (+ 2111us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:53.468471 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:53.468730 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:53.469439 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:53.469451 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:53.470326 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:53.470999 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:53.471024 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.472420 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.472428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:53.474459 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:53.474481 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.475287 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.475297 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:53.475597 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:53.476411 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:53.476444 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:53.477854 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:53.482223 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:53.482229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:53.482233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:53.483046 (+ 813us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:53.483530 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:53.483539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:53.483546 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:53.483906 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:53.484626 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:53.484636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:53.484893 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 01:07:53.485096 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:07:53.485517 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1862,"thread_start_us":1699,"threads_started":1}
I20260806 01:07:53.489413 1868 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:39435
I20260806 01:07:53.489446 2058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:39435 every 8 connection(s)
I20260806 01:07:53.490670 1868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:07:53.492326 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 1868
I20260806 01:07:53.492523 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:07:53.492786 2059 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40555
I20260806 01:07:53.492978 2059 heartbeater.cc:464] Registering TS with master...
I20260806 01:07:53.493124 22781 external_mini_cluster.cc:1428] Setting key a21361776fb78eee697c3251aa07f57c
I20260806 01:07:53.493547 2059 heartbeater.cc:511] Master 127.22.63.126:40555 requested a full tablet report, sending...
I20260806 01:07:53.495082 1359 ts_manager.cc:194] Registered new tserver with Master: 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:53.496129 1359 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:58533
I20260806 01:07:53.496337 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:07:53 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978473, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 01:07:53 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:07:53 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:07:53.916514 1862 heartbeater.cc:503] Master 127.22.63.126:40555 was elected leader, sending a full tablet report...
I20260806 01:07:54.498724 2059 heartbeater.cc:503] Master 127.22.63.126:40555 was elected leader, sending a full tablet report...
W20260806 01:07:54.688416 2070 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.135s user 0.007s sys 0.013s
W20260806 01:07:54.689862 1901 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:53.547892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.22.63.126:40555)
0806 01:07:53.548130 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:53.548138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:53.548289 (+ 151us) server_negotiation.cc:407] Connection header received
0806 01:07:53.548452 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:53.548456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:53.548553 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:53.548652 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:53.550006 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.550728 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.552067 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:53.552525 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:53.552631 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.689469 (+1136838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46968: BlockingRecv error: recv got EOF from 127.0.0.1:46968 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 01:07:54 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978473, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:54.711920 1901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.693935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.22.63.126:40555)
0806 01:07:54.694140 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.694148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.694449 (+ 301us) server_negotiation.cc:407] Connection header received
0806 01:07:54.694659 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.694669 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.694734 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.694859 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:54.696225 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.696964 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.698221 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.698738 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.698890 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.706485 (+ 7595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:54.706491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:54.706494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:54.706533 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:54.709189 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:54.709902 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:54.709912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:54.709914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:54.710044 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:54.710426 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:54.710431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:54.710434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:54.710714 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:54.710989 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.711322 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.711566 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:07:54.716189 1359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:54.723124 1359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:54.758567 2088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.736104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:42469 (local address 127.0.0.1:37414)
0806 01:07:54.747086 (+ 10982us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:54.747165 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:54.747426 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:54.750505 (+ 3079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:54.750513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:54.750564 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:54.751202 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.751224 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.753297 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.753309 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.755303 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.755316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.757391 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.757398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.757611 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.757673 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 01:07:54.757821 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 01:07:54.758048 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10565,"mutex_wait_us":154,"spinlock_wait_cycles":2816,"thread_start_us":12499,"threads_started":1}
I20260806 01:07:54.758662 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.737921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.22.63.67:39435)
0806 01:07:54.749394 (+ 11473us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.749407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.750357 (+ 950us) server_negotiation.cc:407] Connection header received
0806 01:07:54.750471 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.750478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.750692 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.750879 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:54.751725 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.753234 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.756193 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.757570 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.757729 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.757963 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.758168 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.758342 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11290,"thread_start_us":11147,"threads_started":1}
I20260806 01:07:54.758903 2087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.736896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39435 (local address 127.0.0.1:55214)
0806 01:07:54.750087 (+ 13191us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:54.750105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:54.750222 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:54.750899 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:54.750903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:54.750923 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:54.751531 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.751543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.753394 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.753398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.756004 (+ 2606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.756018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.757788 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.757794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.757959 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.758008 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 01:07:54.758120 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 01:07:54.758320 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13074,"mutex_wait_us":104,"thread_start_us":10777,"threads_started":1}
I20260806 01:07:54.759380 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.736546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:34283 (local address 127.0.0.1:38240)
0806 01:07:54.748998 (+ 12452us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:54.749041 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:54.749190 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:54.750518 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:54.750523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:54.750579 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:54.751462 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.751477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.753085 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.753090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.755915 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.755932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.758281 (+ 2349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.758288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.758607 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.758649 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 01:07:54.758758 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 01:07:54.758898 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12249,"mutex_wait_us":473,"spinlock_wait_cycles":18688,"thread_start_us":13280,"threads_started":1}
I20260806 01:07:54.759614 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.737399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.22.63.66:34283)
0806 01:07:54.749860 (+ 12461us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.749868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.749901 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:07:54.750016 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.750026 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.750282 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.750498 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:54.751665 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.752875 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.756159 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.757764 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.757896 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.758097 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.758995 (+ 898us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.759170 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12274,"thread_start_us":12128,"threads_started":1}
I20260806 01:07:54.761032 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.736392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.22.63.65:42469)
0806 01:07:54.749778 (+ 13386us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.749789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.749836 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:07:54.749961 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.749971 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.750256 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.750865 (+ 609us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:54.751457 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.753106 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.755486 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.757131 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.760251 (+ 3120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.760470 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.760553 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.760723 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13217,"thread_start_us":13068,"threads_started":1}
I20260806 01:07:54.763300 1958 tablet_service.cc:1467] Processing CreateTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 (DEFAULT_TABLE table=test-table [id=ccf8010e972245baac2b39a16d36fbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:54.763298 1756 tablet_service.cc:1467] Processing CreateTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 (DEFAULT_TABLE table=test-table [id=ccf8010e972245baac2b39a16d36fbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:54.764657 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c1def1635f4a2ebf067e502d59f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:54.764748 1756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c1def1635f4a2ebf067e502d59f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:54.764660 1564 tablet_service.cc:1467] Processing CreateTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 (DEFAULT_TABLE table=test-table [id=ccf8010e972245baac2b39a16d36fbf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:54.765978 1564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80c1def1635f4a2ebf067e502d59f3b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:54.777133 2094 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Bootstrap starting.
I20260806 01:07:54.779073 2095 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Bootstrap starting.
I20260806 01:07:54.781438 2096 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Bootstrap starting.
I20260806 01:07:54.781817 2094 tablet_bootstrap.cc:654] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:54.782696 2095 tablet_bootstrap.cc:654] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:54.783577 2094 log.cc:824] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:54.784070 2095 log.cc:824] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:54.784938 2096 tablet_bootstrap.cc:654] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:54.785991 2096 log.cc:824] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:07:54.786829 2094 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: No bootstrap required, opened a new log
I20260806 01:07:54.787144 2094 ts_tablet_manager.cc:1397] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 01:07:54.788908 2096 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: No bootstrap required, opened a new log
I20260806 01:07:54.789182 2096 ts_tablet_manager.cc:1397] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:07:54.790354 2095 tablet_bootstrap.cc:492] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: No bootstrap required, opened a new log
I20260806 01:07:54.790660 2095 ts_tablet_manager.cc:1397] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260806 01:07:54.795527 2094 raft_consensus.cc:348] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.795979 2094 raft_consensus.cc:374] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:54.796084 2094 raft_consensus.cc:729] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Initialized, Role: FOLLOWER
I20260806 01:07:54.796751 2094 consensus_queue.cc:260] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.798724 2094 ts_tablet_manager.cc:1428] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260806 01:07:54.800386 2096 raft_consensus.cc:348] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.800832 2096 raft_consensus.cc:374] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:54.800945 2096 raft_consensus.cc:729] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7c70276af41079fa30e7b455ca47e, State: Initialized, Role: FOLLOWER
I20260806 01:07:54.801380 2095 raft_consensus.cc:348] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.801587 2096 consensus_queue.cc:260] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.801791 2095 raft_consensus.cc:374] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:54.801882 2095 raft_consensus.cc:729] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Initialized, Role: FOLLOWER
I20260806 01:07:54.802601 2095 consensus_queue.cc:260] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.803972 2096 ts_tablet_manager.cc:1428] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260806 01:07:54.809011 2095 ts_tablet_manager.cc:1428] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
W20260806 01:07:54.842666 1664 tablet.cc:2369] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:07:54.911883 1863 tablet.cc:2369] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:54.983318 2104 raft_consensus.cc:482] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:54.983680 2104 raft_consensus.cc:504] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.985610 2104 leader_election.cc:288] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
W20260806 01:07:54.993311 2061 tablet.cc:2369] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:07:54.993706 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.986006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:41999 (local address 127.22.63.65:42469)
0806 01:07:54.986249 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.986257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.987174 (+ 917us) server_negotiation.cc:407] Connection header received
0806 01:07:54.987369 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.987374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.987437 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.987556 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:54.988247 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.989230 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.991451 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.992761 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.992950 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.993068 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.993328 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.993456 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:07:54.993785 2107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.986072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:42469 (local address 127.22.63.66:41999)
0806 01:07:54.987046 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:54.987060 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:54.987226 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:54.987578 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:54.987583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:54.987622 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:54.988052 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.988068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.989378 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.989383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.991202 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.991215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.992984 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.992991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.993156 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.993188 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:07:54.993299 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:07:54.993432 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":859,"thread_start_us":493,"threads_started":1}
I20260806 01:07:54.994817 2106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.985953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39435 (local address 127.22.63.66:49933)
0806 01:07:54.987029 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:54.987055 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:54.987881 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:54.988301 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:54.988306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:54.988331 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:54.988747 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.988761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.990153 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.990157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.991967 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:54.991982 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.994255 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.994260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:54.994376 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.994399 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:07:54.994477 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 01:07:54.994560 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"spinlock_wait_cycles":33024,"thread_start_us":587,"threads_started":1}
I20260806 01:07:54.994922 2092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:54.986263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:49933 (local address 127.22.63.67:39435)
0806 01:07:54.986521 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:54.986530 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:54.987170 (+ 640us) server_negotiation.cc:407] Connection header received
0806 01:07:54.988086 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:54.988091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:54.988184 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:54.988280 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:54.988974 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.990010 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.992192 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:54.993546 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:54.993720 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:54.994093 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:54.994545 (+ 452us) server_negotiation.cc:299] Negotiation successful
0806 01:07:54.994688 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:07:54.994833 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c1def1635f4a2ebf067e502d59f3b4" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e" is_pre_election: true
I20260806 01:07:54.995424 1590 raft_consensus.cc:2454] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:54.995843 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c1def1635f4a2ebf067e502d59f3b4" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" is_pre_election: true
I20260806 01:07:54.996115 1690 leader_election.cc:302] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 7bc7c70276af41079fa30e7b455ca47e; no voters:
I20260806 01:07:54.996345 1996 raft_consensus.cc:2454] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:54.996563 2104 raft_consensus.cc:2781] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:54.996711 2104 raft_consensus.cc:482] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:54.996821 2104 raft_consensus.cc:3037] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:54.999091 2104 raft_consensus.cc:504] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:54.999807 2104 leader_election.cc:288] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:55.000208 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c1def1635f4a2ebf067e502d59f3b4" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e"
I20260806 01:07:55.000315 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80c1def1635f4a2ebf067e502d59f3b4" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
I20260806 01:07:55.000425 1590 raft_consensus.cc:3037] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.000516 1996 raft_consensus.cc:3037] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.003698 1590 raft_consensus.cc:2454] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.003698 1996 raft_consensus.cc:2454] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.004142 1690 leader_election.cc:302] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d, 7bc7c70276af41079fa30e7b455ca47e; no voters:
I20260806 01:07:55.004432 2104 raft_consensus.cc:2781] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:55.004863 2104 raft_consensus.cc:686] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 1 LEADER]: Becoming Leader. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Running, Role: LEADER
I20260806 01:07:55.005354 2104 consensus_queue.cc:237] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.010169 1358 catalog_manager.cc:5495] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1121839e62429ba2354609ac5dde20 (127.22.63.66). New cstate: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:55.057796 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.052757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.22.63.66:34283)
0806 01:07:55.052963 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.052967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.052986 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:07:55.053165 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.053168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.053251 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.053382 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:55.053707 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.054394 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.055732 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.056153 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.056246 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.056677 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.057256 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.057372 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.057479 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.057552 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:07:55.066020 1996 raft_consensus.cc:1261] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.066023 1590 raft_consensus.cc:1261] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.067051 2108 consensus_queue.cc:1035] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.067389 2104 consensus_queue.cc:1035] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.079337 2112 mvcc.cc:205] Tried to move back new op lower bound from 7315367833860521984 to 7315367833625714688. Current Snapshot: MvccSnapshot[applied={T|T < 7315367833860521984}]
I20260806 01:07:55.097390 1358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.097775 1358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.101662 1358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Sending DeleteTablet for 3 replicas of tablet 80c1def1635f4a2ebf067e502d59f3b4
I20260806 01:07:55.102885 1756 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260806 01:07:55.102847 1564 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:37414
I20260806 01:07:55.102936 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet 80c1def1635f4a2ebf067e502d59f3b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:55214
I20260806 01:07:55.104703 1359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:55.105630 1359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:55.116714 1563 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d (DEFAULT_TABLE table=test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.117560 1563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa415bd5e0e46c9a6f84413b9c8195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.119243 2135 tablet_replica.cc:330] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: stopping tablet replica
I20260806 01:07:55.119832 2135 raft_consensus.cc:2229] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.120922 1956 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d (DEFAULT_TABLE table=test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.121382 1755 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d (DEFAULT_TABLE table=test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.121575 1956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa415bd5e0e46c9a6f84413b9c8195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.122102 1755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa415bd5e0e46c9a6f84413b9c8195d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.125998 2096 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Bootstrap starting.
I20260806 01:07:55.127952 2096 tablet_bootstrap.cc:654] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.128652 2094 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Bootstrap starting.
I20260806 01:07:55.130235 2094 tablet_bootstrap.cc:654] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.131342 2095 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Bootstrap starting.
I20260806 01:07:55.132071 2135 raft_consensus.cc:2258] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.133420 2095 tablet_bootstrap.cc:654] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.133822 2094 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: No bootstrap required, opened a new log
I20260806 01:07:55.133977 2094 ts_tablet_manager.cc:1397] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 01:07:55.134891 2094 raft_consensus.cc:348] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.135130 2094 raft_consensus.cc:374] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.135231 2094 raft_consensus.cc:729] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.135478 2135 ts_tablet_manager.cc:1905] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.135499 2094 consensus_queue.cc:260] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.136092 2094 ts_tablet_manager.cc:1428] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:07:55.138700 2095 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: No bootstrap required, opened a new log
I20260806 01:07:55.138855 2095 ts_tablet_manager.cc:1397] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:07:55.139853 2133 tablet_replica.cc:330] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: stopping tablet replica
I20260806 01:07:55.140349 2133 raft_consensus.cc:2229] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:55.140913 2095 raft_consensus.cc:348] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.141131 2133 raft_consensus.cc:2258] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.141165 2095 raft_consensus.cc:374] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.141314 2095 raft_consensus.cc:729] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.141356 2096 tablet_bootstrap.cc:492] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: No bootstrap required, opened a new log
I20260806 01:07:55.141507 2096 ts_tablet_manager.cc:1397] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260806 01:07:55.141633 2095 consensus_queue.cc:260] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.141952 2134 tablet_replica.cc:330] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: stopping tablet replica
I20260806 01:07:55.142367 2134 raft_consensus.cc:2229] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.142459 2096 raft_consensus.cc:348] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.142689 2096 raft_consensus.cc:374] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.142807 2096 raft_consensus.cc:729] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7c70276af41079fa30e7b455ca47e, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.142872 2134 raft_consensus.cc:2258] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.143219 2096 consensus_queue.cc:260] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.144845 2096 ts_tablet_manager.cc:1428] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:55.145630 2095 ts_tablet_manager.cc:1428] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260806 01:07:55.149392 2135 ts_tablet_manager.cc:1918] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.149578 2135 log.cc:1196] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/80c1def1635f4a2ebf067e502d59f3b4
I20260806 01:07:55.150189 2135 ts_tablet_manager.cc:1939] T 80c1def1635f4a2ebf067e502d59f3b4 P 7bc7c70276af41079fa30e7b455ca47e: Deleting consensus metadata
I20260806 01:07:55.151775 1336 catalog_manager.cc:4841] TS 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469): tablet 80c1def1635f4a2ebf067e502d59f3b4 (table test-table [id=ccf8010e972245baac2b39a16d36fbf7]) successfully deleted
I20260806 01:07:55.153167 2134 ts_tablet_manager.cc:1905] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.154871 2133 ts_tablet_manager.cc:1905] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.156888 2108 raft_consensus.cc:482] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:55.157071 2108 raft_consensus.cc:504] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.158285 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa415bd5e0e46c9a6f84413b9c8195d" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" is_pre_election: true
I20260806 01:07:55.158541 1996 raft_consensus.cc:2454] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:55.159245 1689 leader_election.cc:302] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d; no voters:
I20260806 01:07:55.159507 2134 ts_tablet_manager.cc:1918] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.159655 2134 log.cc:1196] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/80c1def1635f4a2ebf067e502d59f3b4
I20260806 01:07:55.159621 2124 raft_consensus.cc:2781] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:55.159788 2124 raft_consensus.cc:482] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:55.159933 2124 raft_consensus.cc:3037] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.159842 2108 leader_election.cc:288] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:55.160243 2134 ts_tablet_manager.cc:1939] T 80c1def1635f4a2ebf067e502d59f3b4 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting consensus metadata
I20260806 01:07:55.160421 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa415bd5e0e46c9a6f84413b9c8195d" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e" is_pre_election: true
I20260806 01:07:55.160720 1590 raft_consensus.cc:2454] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:55.162221 1335 catalog_manager.cc:4841] TS 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435): tablet 80c1def1635f4a2ebf067e502d59f3b4 (table test-table [id=ccf8010e972245baac2b39a16d36fbf7]) successfully deleted
I20260806 01:07:55.163035 2124 raft_consensus.cc:504] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.164242 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa415bd5e0e46c9a6f84413b9c8195d" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e"
I20260806 01:07:55.164359 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa415bd5e0e46c9a6f84413b9c8195d" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
I20260806 01:07:55.164458 1590 raft_consensus.cc:3037] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.164578 1996 raft_consensus.cc:3037] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.166850 2133 ts_tablet_manager.cc:1918] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.167090 2133 log.cc:1196] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/80c1def1635f4a2ebf067e502d59f3b4
I20260806 01:07:55.167421 1590 raft_consensus.cc:2454] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.167452 1996 raft_consensus.cc:2454] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.167868 2133 ts_tablet_manager.cc:1939] T 80c1def1635f4a2ebf067e502d59f3b4 P 2d1121839e62429ba2354609ac5dde20: Deleting consensus metadata
I20260806 01:07:55.167899 1689 leader_election.cc:302] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d; no voters:
I20260806 01:07:55.168126 2124 leader_election.cc:288] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:55.168453 2124 raft_consensus.cc:2781] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:55.168627 2124 raft_consensus.cc:686] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 1 LEADER]: Becoming Leader. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Running, Role: LEADER
I20260806 01:07:55.168900 2124 consensus_queue.cc:237] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.169869 1337 catalog_manager.cc:4841] TS 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283): tablet 80c1def1635f4a2ebf067e502d59f3b4 (table test-table [id=ccf8010e972245baac2b39a16d36fbf7]) successfully deleted
I20260806 01:07:55.172484 1357 catalog_manager.cc:5495] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1121839e62429ba2354609ac5dde20 (127.22.63.66). New cstate: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } health_report { overall_health: UNKNOWN } } }
Aug 06 01:07:55 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978471, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.126@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:55.230614 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.214419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.22.63.126:40555)
0806 01:07:55.215354 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.215358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.215378 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:07:55.215422 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.215425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.215482 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.215571 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:55.217162 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.217988 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.219898 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.220324 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.220442 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.224623 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:55.224629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:55.224632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:55.224662 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:55.227495 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:55.228445 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:55.228450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:55.228453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:55.228607 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:55.229066 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:55.229070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:55.229072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:55.229352 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:55.229484 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.229974 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.230320 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":838,"thread_start_us":651,"threads_started":1}
I20260806 01:07:55.230659 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.214345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40555 (local address 127.0.0.1:46984)
0806 01:07:55.214982 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:55.215015 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:55.215167 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:55.215644 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:55.215649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:55.216641 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:07:55.216984 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.217012 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.218122 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.218128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.219697 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.219714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.220503 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.220509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.220727 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.221451 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:07:55.221482 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:07:55.224400 (+ 2918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:07:55.227632 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:55.227646 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:55.227654 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:55.228300 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:55.228724 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:07:55.228729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:07:55.228733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:07:55.228913 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:07:55.229488 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:07:55.229498 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:07:55.229695 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 01:07:55.229873 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 01:07:55.230181 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":251,"threads_started":1}
I20260806 01:07:55.236263 1357 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46984:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 01:07:55.237668 1357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:55.245721 1756 tablet_service.cc:1467] Processing CreateTablet for tablet bcb9a9a39034404db9af2ddf923db5cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd490758517e4f9384bf7f97300d4f6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:55.245823 1958 tablet_service.cc:1467] Processing CreateTablet for tablet bcb9a9a39034404db9af2ddf923db5cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd490758517e4f9384bf7f97300d4f6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:55.246060 1564 tablet_service.cc:1467] Processing CreateTablet for tablet bcb9a9a39034404db9af2ddf923db5cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd490758517e4f9384bf7f97300d4f6a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 01:07:55.246392 1756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb9a9a39034404db9af2ddf923db5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.246414 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb9a9a39034404db9af2ddf923db5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.246629 1564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcb9a9a39034404db9af2ddf923db5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.253808 2094 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: Bootstrap starting.
I20260806 01:07:55.253937 2095 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: Bootstrap starting.
I20260806 01:07:55.254585 2096 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: Bootstrap starting.
I20260806 01:07:55.255578 2094 tablet_bootstrap.cc:654] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.255723 2095 tablet_bootstrap.cc:654] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.256323 2096 tablet_bootstrap.cc:654] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.268039 2095 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: No bootstrap required, opened a new log
I20260806 01:07:55.268042 2096 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: No bootstrap required, opened a new log
I20260806 01:07:55.268247 2095 ts_tablet_manager.cc:1397] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260806 01:07:55.268453 2096 ts_tablet_manager.cc:1397] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.001s
I20260806 01:07:55.269066 2095 raft_consensus.cc:348] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.269335 2095 raft_consensus.cc:374] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.269421 2095 raft_consensus.cc:729] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.269708 2095 consensus_queue.cc:260] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.270197 2124 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.270473 2124 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.271021 2095 ts_tablet_manager.cc:1428] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 01:07:55.269718 2096 raft_consensus.cc:348] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.271402 2096 raft_consensus.cc:374] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.271543 2096 raft_consensus.cc:729] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7c70276af41079fa30e7b455ca47e, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.271937 2096 consensus_queue.cc:260] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.272563 2103 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.273029 2103 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.276522 2096 ts_tablet_manager.cc:1428] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260806 01:07:55.278918 2094 tablet_bootstrap.cc:492] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: No bootstrap required, opened a new log
I20260806 01:07:55.279148 2094 ts_tablet_manager.cc:1397] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.009s
I20260806 01:07:55.279861 2094 raft_consensus.cc:348] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.280082 2094 raft_consensus.cc:374] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.280179 2094 raft_consensus.cc:729] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.280478 2094 consensus_queue.cc:260] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.281134 2094 ts_tablet_manager.cc:1428] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 01:07:55.280934 2102 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.281332 2102 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.428684 2124 raft_consensus.cc:482] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:55.428941 2124 raft_consensus.cc:504] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.429764 2124 leader_election.cc:288] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:55.430383 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb9a9a39034404db9af2ddf923db5cf" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e" is_pre_election: true
I20260806 01:07:55.430394 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb9a9a39034404db9af2ddf923db5cf" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" is_pre_election: true
I20260806 01:07:55.430684 1590 raft_consensus.cc:2454] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:55.430675 1996 raft_consensus.cc:2454] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 0.
I20260806 01:07:55.431187 1689 leader_election.cc:302] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d; no voters:
I20260806 01:07:55.431517 2124 raft_consensus.cc:2781] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:55.431659 2124 raft_consensus.cc:482] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:55.431807 2124 raft_consensus.cc:3037] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.435240 2124 raft_consensus.cc:504] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.436002 2124 leader_election.cc:288] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Requested vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435)
I20260806 01:07:55.436555 1590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb9a9a39034404db9af2ddf923db5cf" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e"
I20260806 01:07:55.436566 1996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcb9a9a39034404db9af2ddf923db5cf" candidate_uuid: "2d1121839e62429ba2354609ac5dde20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54254e1c5b484ca7aaddcf1836d4e15d"
I20260806 01:07:55.436818 1590 raft_consensus.cc:3037] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.436812 1996 raft_consensus.cc:3037] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.438848 1590 raft_consensus.cc:2454] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.439337 1690 leader_election.cc:302] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 7bc7c70276af41079fa30e7b455ca47e; no voters:
I20260806 01:07:55.439639 2124 raft_consensus.cc:2781] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:55.439868 2124 raft_consensus.cc:686] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [term 1 LEADER]: Becoming Leader. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Running, Role: LEADER
I20260806 01:07:55.439867 1996 raft_consensus.cc:2454] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d1121839e62429ba2354609ac5dde20 in term 1.
I20260806 01:07:55.440160 2124 consensus_queue.cc:237] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.442268 2108 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: TxnStatusTablet state changed. Reason: New leader 2d1121839e62429ba2354609ac5dde20. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.442656 2108 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:55.443354 2163 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 01:07:55.443629 2163 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 01:07:55.444295 1359 catalog_manager.cc:5495] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d1121839e62429ba2354609ac5dde20 (127.22.63.66). New cstate: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } health_report { overall_health: UNKNOWN } } }
I20260806 01:07:55.468709 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.463427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:34283 (local address 127.0.0.1:38258)
0806 01:07:55.463665 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:55.463691 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:55.463898 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:55.464347 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:55.464353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:55.464365 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:55.464561 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.464570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.465639 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.465646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.466648 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.466659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.467418 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.467424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.467614 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.467675 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.468148 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.468172 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:07:55.468257 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 01:07:55.468413 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 01:07:55.468735 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.463651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38258 (local address 127.22.63.66:34283)
0806 01:07:55.463886 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.463893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.463914 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:07:55.464117 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.464122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.464215 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.464317 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:55.464693 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.465499 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.466812 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.467250 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.467424 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.467906 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.468040 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.468146 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.468292 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.468375 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 01:07:55.472568 1996 raft_consensus.cc:1261] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.472657 1590 raft_consensus.cc:1261] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.473275 2108 consensus_queue.cc:1035] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.473839 2124 consensus_queue.cc:1035] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.477531 2103 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: TxnStatusTablet state changed. Reason: New leader 2d1121839e62429ba2354609ac5dde20. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.477898 2103 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.479887 2102 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: TxnStatusTablet state changed. Reason: New leader 2d1121839e62429ba2354609ac5dde20. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.480237 2102 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.480718 2108 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.481009 2108 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:55.483695 2103 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.483985 2103 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 7bc7c70276af41079fa30e7b455ca47e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.486649 2124 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.487126 2124 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 2d1121839e62429ba2354609ac5dde20: This TxnStatusTablet replica's current role is: LEADER
I20260806 01:07:55.489566 2102 tablet_replica.cc:439] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d1121839e62429ba2354609ac5dde20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } } }
I20260806 01:07:55.489845 2102 tablet_replica.cc:442] T bcb9a9a39034404db9af2ddf923db5cf P 54254e1c5b484ca7aaddcf1836d4e15d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 01:07:55.513185 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.505398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59725 (local address 127.22.63.66:34283)
0806 01:07:55.505596 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.505603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.506245 (+ 642us) server_negotiation.cc:407] Connection header received
0806 01:07:55.506486 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.506491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.506583 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.506676 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:55.507261 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.508448 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.510684 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.511422 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.511576 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.512191 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.512339 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.512428 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.512614 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.512849 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":62}
I20260806 01:07:55.513356 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.505343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:34283 (local address 127.22.63.66:59725)
0806 01:07:55.506124 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:55.506145 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:55.506336 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:55.506723 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:55.506729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:55.506743 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:55.507084 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.507097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.508597 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.508609 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.510510 (+ 1901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.510532 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.511695 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.511702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.511908 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.511984 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.512461 (+ 477us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.512474 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:07:55.512578 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 01:07:55.512891 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":51,"thread_start_us":421,"threads_started":1}
I20260806 01:07:55.524291 1996 raft_consensus.cc:1261] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.524292 1590 raft_consensus.cc:1261] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Refusing update from remote peer 2d1121839e62429ba2354609ac5dde20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.524916 2108 consensus_queue.cc:1035] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.525413 2108 consensus_queue.cc:1035] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.540344 2116 mvcc.cc:205] Tried to move back new op lower bound from 7315367835742334976 to 7315367834294632448. Current Snapshot: MvccSnapshot[applied={T|T < 7315367835742334976}]
I20260806 01:07:55.593429 1359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.593721 1359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.597221 1359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Sending DeleteTablet for 3 replicas of tablet 2aa415bd5e0e46c9a6f84413b9c8195d
I20260806 01:07:55.598130 1756 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260806 01:07:55.598179 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:55214
I20260806 01:07:55.598343 1564 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa415bd5e0e46c9a6f84413b9c8195d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:37414
I20260806 01:07:55.598451 2133 tablet_replica.cc:330] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: stopping tablet replica
I20260806 01:07:55.598639 2135 tablet_replica.cc:330] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: stopping tablet replica
I20260806 01:07:55.598673 2133 raft_consensus.cc:2229] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:55.598834 2135 raft_consensus.cc:2229] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.599035 2133 raft_consensus.cc:2258] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.599059 2135 raft_consensus.cc:2258] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.599323 2134 tablet_replica.cc:330] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: stopping tablet replica
I20260806 01:07:55.599541 2134 raft_consensus.cc:2229] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.599757 2134 raft_consensus.cc:2258] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.600340 2133 ts_tablet_manager.cc:1905] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.600471 2135 ts_tablet_manager.cc:1905] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.607673 2135 ts_tablet_manager.cc:1918] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:55.607842 2135 log.cc:1196] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/2aa415bd5e0e46c9a6f84413b9c8195d
I20260806 01:07:55.608261 2135 ts_tablet_manager.cc:1939] T 2aa415bd5e0e46c9a6f84413b9c8195d P 7bc7c70276af41079fa30e7b455ca47e: Deleting consensus metadata
I20260806 01:07:55.608685 2133 ts_tablet_manager.cc:1918] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:55.608867 2133 log.cc:1196] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/2aa415bd5e0e46c9a6f84413b9c8195d
I20260806 01:07:55.609185 2134 ts_tablet_manager.cc:1905] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.609320 2133 ts_tablet_manager.cc:1939] T 2aa415bd5e0e46c9a6f84413b9c8195d P 2d1121839e62429ba2354609ac5dde20: Deleting consensus metadata
I20260806 01:07:55.609297 1336 catalog_manager.cc:4841] TS 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469): tablet 2aa415bd5e0e46c9a6f84413b9c8195d (table test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]) successfully deleted
I20260806 01:07:55.610386 1337 catalog_manager.cc:4841] TS 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283): tablet 2aa415bd5e0e46c9a6f84413b9c8195d (table test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]) successfully deleted
I20260806 01:07:55.614667 2134 ts_tablet_manager.cc:1918] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:55.614858 2134 log.cc:1196] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/2aa415bd5e0e46c9a6f84413b9c8195d
I20260806 01:07:55.615227 2134 ts_tablet_manager.cc:1939] T 2aa415bd5e0e46c9a6f84413b9c8195d P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting consensus metadata
I20260806 01:07:55.616186 1335 catalog_manager.cc:4841] TS 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435): tablet 2aa415bd5e0e46c9a6f84413b9c8195d (table test-table [id=1f0d83b1c8b2413db1092b1fb3dc1ac5]) successfully deleted
Aug 06 01:07:55 dist-test-slave-7f01 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 01:07:55 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:07:55 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:07:55.681619 2153 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:07:55.669769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.22.63.126:40555)
0806 01:07:55.670048 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.670056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.670422 (+ 366us) server_negotiation.cc:407] Connection header received
0806 01:07:55.670650 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.670654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.670724 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.670841 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:55.672384 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.673139 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.674380 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.674795 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.674934 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.681339 (+ 6405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46992: BlockingRecv error: recv got EOF from 127.0.0.1:46992 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 01:07:55 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM
I20260806 01:07:55.701063 2153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.684868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.22.63.126:40555)
0806 01:07:55.685111 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.685120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.685658 (+ 538us) server_negotiation.cc:407] Connection header received
0806 01:07:55.685872 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.685878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.685955 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.686065 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:55.687613 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.688433 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.689678 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.690125 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.690259 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.694286 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:55.694304 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:55.694309 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:55.694354 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:55.698111 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:55.698746 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:55.698752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:55.698756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:55.698946 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:55.699382 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:55.699388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:55.699394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:55.699821 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:55.700106 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.700422 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.700666 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:07:55.704694 1359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:07:55.705353 1359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:55.712116 1564 tablet_service.cc:1467] Processing CreateTablet for tablet e6b48768b9944e2b8ded027114ad92a9 (DEFAULT_TABLE table=test-table [id=a4abf48583714fd9a5f596d15f61db4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.712647 1564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6b48768b9944e2b8ded027114ad92a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.712749 1958 tablet_service.cc:1467] Processing CreateTablet for tablet e6b48768b9944e2b8ded027114ad92a9 (DEFAULT_TABLE table=test-table [id=a4abf48583714fd9a5f596d15f61db4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.713178 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6b48768b9944e2b8ded027114ad92a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.716288 1756 tablet_service.cc:1467] Processing CreateTablet for tablet e6b48768b9944e2b8ded027114ad92a9 (DEFAULT_TABLE table=test-table [id=a4abf48583714fd9a5f596d15f61db4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.716799 1756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6b48768b9944e2b8ded027114ad92a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.719218 2096 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Bootstrap starting.
I20260806 01:07:55.719424 2094 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Bootstrap starting.
I20260806 01:07:55.720997 2096 tablet_bootstrap.cc:654] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.721012 2094 tablet_bootstrap.cc:654] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.724287 2095 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Bootstrap starting.
I20260806 01:07:55.726011 2095 tablet_bootstrap.cc:654] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.732224 2094 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: No bootstrap required, opened a new log
I20260806 01:07:55.732426 2094 ts_tablet_manager.cc:1397] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260806 01:07:55.733546 2094 raft_consensus.cc:348] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.734012 2094 raft_consensus.cc:374] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.734139 2094 raft_consensus.cc:729] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.734473 2094 consensus_queue.cc:260] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.735211 2094 ts_tablet_manager.cc:1428] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:55.736776 2095 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: No bootstrap required, opened a new log
I20260806 01:07:55.737011 2095 ts_tablet_manager.cc:1397] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260806 01:07:55.737857 2095 raft_consensus.cc:348] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.738168 2096 tablet_bootstrap.cc:492] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: No bootstrap required, opened a new log
I20260806 01:07:55.738217 2095 raft_consensus.cc:374] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.738336 2096 ts_tablet_manager.cc:1397] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.010s
I20260806 01:07:55.738444 2095 raft_consensus.cc:729] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.738773 2095 consensus_queue.cc:260] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.739207 2096 raft_consensus.cc:348] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.739480 2096 raft_consensus.cc:374] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.739588 2096 raft_consensus.cc:729] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7c70276af41079fa30e7b455ca47e, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.739868 2096 consensus_queue.cc:260] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.740371 2095 ts_tablet_manager.cc:1428] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 01:07:55.740995 2096 ts_tablet_manager.cc:1428] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 01:07:55.840133 2102 raft_consensus.cc:482] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:55.840421 2102 raft_consensus.cc:504] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.842051 2102 leader_election.cc:288] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283)
I20260806 01:07:55.850844 2202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.842803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:35411 (local address 127.22.63.65:42469)
0806 01:07:55.843961 (+ 1158us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.843968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.844001 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:07:55.844077 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.844083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.844153 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.844260 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:55.845052 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.846109 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.848682 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.849645 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.849890 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.849981 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.850255 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.850409 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":939,"threads_started":1}
I20260806 01:07:55.850890 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.842323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:42469 (local address 127.22.63.67:35411)
0806 01:07:55.843272 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:55.843292 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:55.843475 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:55.844323 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:55.844330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:55.844385 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:55.844776 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.844790 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.846281 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.846287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.848486 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.848502 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.849866 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.849875 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.850069 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.850096 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:07:55.850214 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 01:07:55.850382 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"spinlock_wait_cycles":41344,"thread_start_us":1706,"threads_started":1}
I20260806 01:07:55.851418 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.842672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:45197 (local address 127.22.63.66:34283)
0806 01:07:55.842873 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.842879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.844507 (+ 1628us) server_negotiation.cc:407] Connection header received
0806 01:07:55.844724 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.844734 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.844819 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.844920 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:07:55.845776 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.846802 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.848799 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.850180 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.850361 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.850428 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.850997 (+ 569us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.851141 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:07:55.851939 1589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b48768b9944e2b8ded027114ad92a9" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e" is_pre_election: true
I20260806 01:07:55.852137 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.842342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:34283 (local address 127.22.63.67:45197)
0806 01:07:55.844358 (+ 2016us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:55.844384 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:55.844560 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:55.845118 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:55.845124 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:55.845160 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:07:55.845572 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.845583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.846966 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.846975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.848621 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:55.848641 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.850594 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.850601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:55.850779 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.850812 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:07:55.851648 (+ 836us) client_negotiation.cc:240] Negotiation successful
0806 01:07:55.851847 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1884,"spinlock_wait_cycles":7296,"thread_start_us":676,"threads_started":1}
I20260806 01:07:55.852288 1589 raft_consensus.cc:2454] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 0.
I20260806 01:07:55.852952 1892 leader_election.cc:302] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54254e1c5b484ca7aaddcf1836d4e15d, 7bc7c70276af41079fa30e7b455ca47e; no voters:
I20260806 01:07:55.853433 2102 raft_consensus.cc:2781] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:55.853549 2102 raft_consensus.cc:482] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:55.853646 2102 raft_consensus.cc:3037] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.853353 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b48768b9944e2b8ded027114ad92a9" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1121839e62429ba2354609ac5dde20" is_pre_election: true
I20260806 01:07:55.853940 1790 raft_consensus.cc:2454] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 0.
I20260806 01:07:55.856480 2102 raft_consensus.cc:504] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.857251 2102 leader_election.cc:288] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 election: Requested vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283)
I20260806 01:07:55.857755 1589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b48768b9944e2b8ded027114ad92a9" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e"
I20260806 01:07:55.857820 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6b48768b9944e2b8ded027114ad92a9" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1121839e62429ba2354609ac5dde20"
I20260806 01:07:55.858016 1589 raft_consensus.cc:3037] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.858044 1790 raft_consensus.cc:3037] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:55.861166 1589 raft_consensus.cc:2454] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 1.
I20260806 01:07:55.861619 1790 raft_consensus.cc:2454] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 1.
I20260806 01:07:55.861696 1892 leader_election.cc:302] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54254e1c5b484ca7aaddcf1836d4e15d, 7bc7c70276af41079fa30e7b455ca47e; no voters:
I20260806 01:07:55.862015 2102 raft_consensus.cc:2781] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:55.862562 2102 raft_consensus.cc:686] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 LEADER]: Becoming Leader. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Running, Role: LEADER
I20260806 01:07:55.863046 2102 consensus_queue.cc:237] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.866607 1358 catalog_manager.cc:5495] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67). New cstate: current_term: 1 leader_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } health_report { overall_health: HEALTHY } } }
I20260806 01:07:55.898610 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:55.891435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55230 (local address 127.22.63.67:39435)
0806 01:07:55.892252 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:55.892260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:55.892290 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:07:55.892415 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:55.892421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:55.892552 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:55.892755 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:55.893120 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.894324 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.895679 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:55.896363 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:55.896496 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:55.896977 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:55.897828 (+ 851us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:55.897999 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:55.898109 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 01:07:55.898200 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":407,"threads_started":1}
I20260806 01:07:55.902892 1790 raft_consensus.cc:1261] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Refusing update from remote peer 54254e1c5b484ca7aaddcf1836d4e15d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.903023 1589 raft_consensus.cc:1261] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Refusing update from remote peer 54254e1c5b484ca7aaddcf1836d4e15d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:55.903856 2205 consensus_queue.cc:1035] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.904242 2102 consensus_queue.cc:1035] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:55.916529 2113 mvcc.cc:205] Tried to move back new op lower bound from 7315367837289971712 to 7315367837139337216. Current Snapshot: MvccSnapshot[applied={T|T < 7315367837289971712 or (T in {7315367837289971712})}]
I20260806 01:07:55.938182 1359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.938532 1359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:55.942256 1359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Sending DeleteTablet for 3 replicas of tablet e6b48768b9944e2b8ded027114ad92a9
I20260806 01:07:55.944444 1564 tablet_service.cc:1514] Processing DeleteTablet for tablet e6b48768b9944e2b8ded027114ad92a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:37414
I20260806 01:07:55.944813 1756 tablet_service.cc:1514] Processing DeleteTablet for tablet e6b48768b9944e2b8ded027114ad92a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260806 01:07:55.945734 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet e6b48768b9944e2b8ded027114ad92a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:55 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:55214
I20260806 01:07:55.946108 2135 tablet_replica.cc:330] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: stopping tablet replica
I20260806 01:07:55.946300 2135 raft_consensus.cc:2229] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.946548 2135 raft_consensus.cc:2258] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.946964 2134 tablet_replica.cc:330] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: stopping tablet replica
I20260806 01:07:55.947130 2134 raft_consensus.cc:2229] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:55.947660 2134 raft_consensus.cc:2258] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.947700 2133 tablet_replica.cc:330] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: stopping tablet replica
I20260806 01:07:55.948264 1359 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 01:07:55.948870 2135 ts_tablet_manager.cc:1905] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.948879 2134 ts_tablet_manager.cc:1905] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 01:07:55.949004 1359 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:07:55.951853 2133 raft_consensus.cc:2229] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:55.952277 2133 raft_consensus.cc:2258] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:55.955926 2135 ts_tablet_manager.cc:1918] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.956058 2133 ts_tablet_manager.cc:1905] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:55.956090 2135 log.cc:1196] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/e6b48768b9944e2b8ded027114ad92a9
I20260806 01:07:55.956562 2135 ts_tablet_manager.cc:1939] T e6b48768b9944e2b8ded027114ad92a9 P 7bc7c70276af41079fa30e7b455ca47e: Deleting consensus metadata
I20260806 01:07:55.959554 1336 catalog_manager.cc:4841] TS 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469): tablet e6b48768b9944e2b8ded027114ad92a9 (table test-table [id=a4abf48583714fd9a5f596d15f61db4d]) successfully deleted
I20260806 01:07:55.963810 2134 ts_tablet_manager.cc:1918] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.964082 2134 log.cc:1196] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/e6b48768b9944e2b8ded027114ad92a9
I20260806 01:07:55.964689 1958 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (DEFAULT_TABLE table=test-table [id=2f807bb785c04633a74e09308f6949b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.965167 1958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7625ec496d46bdbfc2cbe3e2093a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.965054 1756 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (DEFAULT_TABLE table=test-table [id=2f807bb785c04633a74e09308f6949b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.965488 1756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7625ec496d46bdbfc2cbe3e2093a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.972405 2094 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Bootstrap starting.
I20260806 01:07:55.972699 1564 tablet_service.cc:1467] Processing CreateTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (DEFAULT_TABLE table=test-table [id=2f807bb785c04633a74e09308f6949b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:07:55.973196 1564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f7625ec496d46bdbfc2cbe3e2093a81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:07:55.974320 2094 tablet_bootstrap.cc:654] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.964689 2134 ts_tablet_manager.cc:1939] T e6b48768b9944e2b8ded027114ad92a9 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting consensus metadata
I20260806 01:07:55.980130 1335 catalog_manager.cc:4841] TS 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435): tablet e6b48768b9944e2b8ded027114ad92a9 (table test-table [id=a4abf48583714fd9a5f596d15f61db4d]) successfully deleted
I20260806 01:07:55.984922 2094 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: No bootstrap required, opened a new log
I20260806 01:07:55.985198 2094 ts_tablet_manager.cc:1397] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260806 01:07:55.986151 2094 raft_consensus.cc:348] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.986543 2094 raft_consensus.cc:374] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:55.986701 2094 raft_consensus.cc:729] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Initialized, Role: FOLLOWER
I20260806 01:07:55.987097 2094 consensus_queue.cc:260] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:55.988631 2094 ts_tablet_manager.cc:1428] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260806 01:07:55.991068 2133 ts_tablet_manager.cc:1918] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:07:55.993109 2096 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Bootstrap starting.
I20260806 01:07:55.994973 2096 tablet_bootstrap.cc:654] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.992002 2095 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Bootstrap starting.
I20260806 01:07:55.996783 2133 log.cc:1196] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/e6b48768b9944e2b8ded027114ad92a9
I20260806 01:07:55.997462 2133 ts_tablet_manager.cc:1939] T e6b48768b9944e2b8ded027114ad92a9 P 2d1121839e62429ba2354609ac5dde20: Deleting consensus metadata
I20260806 01:07:55.998692 2095 tablet_bootstrap.cc:654] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Neither blocks nor log segments found. Creating new log.
I20260806 01:07:55.999971 1337 catalog_manager.cc:4841] TS 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283): tablet e6b48768b9944e2b8ded027114ad92a9 (table test-table [id=a4abf48583714fd9a5f596d15f61db4d]) successfully deleted
I20260806 01:07:56.000505 2096 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: No bootstrap required, opened a new log
I20260806 01:07:56.000686 2096 ts_tablet_manager.cc:1397] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260806 01:07:56.001587 2096 raft_consensus.cc:348] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.001935 2096 raft_consensus.cc:374] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:56.002019 2096 raft_consensus.cc:729] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7c70276af41079fa30e7b455ca47e, State: Initialized, Role: FOLLOWER
I20260806 01:07:56.002305 2096 consensus_queue.cc:260] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.003929 2096 ts_tablet_manager.cc:1428] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 01:07:56.009985 2095 tablet_bootstrap.cc:492] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: No bootstrap required, opened a new log
I20260806 01:07:56.010227 2095 ts_tablet_manager.cc:1397] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20260806 01:07:56.010922 2095 raft_consensus.cc:348] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.011180 2095 raft_consensus.cc:374] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:07:56.011277 2095 raft_consensus.cc:729] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1121839e62429ba2354609ac5dde20, State: Initialized, Role: FOLLOWER
I20260806 01:07:56.011546 2095 consensus_queue.cc:260] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.012176 2095 ts_tablet_manager.cc:1428] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 01:07:56.191005 2102 raft_consensus.cc:482] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:07:56.191219 2102 raft_consensus.cc:504] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.192214 2102 leader_election.cc:288] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283)
I20260806 01:07:56.192664 1589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7625ec496d46bdbfc2cbe3e2093a81" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e" is_pre_election: true
I20260806 01:07:56.192723 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7625ec496d46bdbfc2cbe3e2093a81" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1121839e62429ba2354609ac5dde20" is_pre_election: true
I20260806 01:07:56.192958 1589 raft_consensus.cc:2454] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 0.
I20260806 01:07:56.192955 1790 raft_consensus.cc:2454] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 0.
I20260806 01:07:56.193456 1893 leader_election.cc:302] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d; no voters:
I20260806 01:07:56.193819 2102 raft_consensus.cc:2781] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:07:56.193959 2102 raft_consensus.cc:482] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:07:56.194077 2102 raft_consensus.cc:3037] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:56.197372 2102 raft_consensus.cc:504] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.198105 2102 leader_election.cc:288] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 election: Requested vote from peers 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469), 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283)
I20260806 01:07:56.198593 1790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7625ec496d46bdbfc2cbe3e2093a81" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d1121839e62429ba2354609ac5dde20"
I20260806 01:07:56.198645 1589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f7625ec496d46bdbfc2cbe3e2093a81" candidate_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bc7c70276af41079fa30e7b455ca47e"
I20260806 01:07:56.198849 1790 raft_consensus.cc:3037] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:56.198868 1589 raft_consensus.cc:3037] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:07:56.201153 1790 raft_consensus.cc:2454] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 1.
I20260806 01:07:56.201592 1893 leader_election.cc:302] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d1121839e62429ba2354609ac5dde20, 54254e1c5b484ca7aaddcf1836d4e15d; no voters:
I20260806 01:07:56.201902 2102 raft_consensus.cc:2781] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:07:56.202087 2102 raft_consensus.cc:686] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 LEADER]: Becoming Leader. State: Replica: 54254e1c5b484ca7aaddcf1836d4e15d, State: Running, Role: LEADER
I20260806 01:07:56.202373 1589 raft_consensus.cc:2454] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54254e1c5b484ca7aaddcf1836d4e15d in term 1.
I20260806 01:07:56.202385 2102 consensus_queue.cc:237] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } }
I20260806 01:07:56.205497 1359 catalog_manager.cc:5495] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d reported cstate change: term changed from 0 to 1, leader changed from <none> to 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67). New cstate: current_term: 1 leader_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54254e1c5b484ca7aaddcf1836d4e15d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 39435 } health_report { overall_health: HEALTHY } } }
I20260806 01:07:56.258844 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.252586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:34283 (local address 127.22.63.67:46477)
0806 01:07:56.253139 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:56.253155 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:56.253281 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:56.253665 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:56.253669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:56.253678 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:56.253994 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:56.254003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.255465 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.255468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:56.256723 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:56.256735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.257709 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.257717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:56.257858 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.257912 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:56.258350 (+ 438us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:56.258358 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:07:56.258429 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 01:07:56.258528 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":378,"threads_started":1}
I20260806 01:07:56.258908 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.252702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:46477 (local address 127.22.63.66:34283)
0806 01:07:56.252939 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:56.252947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:56.253250 (+ 303us) server_negotiation.cc:407] Connection header received
0806 01:07:56.253420 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:56.253427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:56.253532 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:56.253638 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:56.254154 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.255328 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.256877 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.257529 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.257676 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.258094 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:56.258243 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:56.258349 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:56.258510 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 01:07:56.258596 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:07:56.268509 1589 raft_consensus.cc:1261] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Refusing update from remote peer 54254e1c5b484ca7aaddcf1836d4e15d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:56.268509 1790 raft_consensus.cc:1261] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Refusing update from remote peer 54254e1c5b484ca7aaddcf1836d4e15d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:07:56.269114 2102 consensus_queue.cc:1035] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d1121839e62429ba2354609ac5dde20" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:56.269542 2205 consensus_queue.cc:1035] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bc7c70276af41079fa30e7b455ca47e" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 42469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:07:56.314985 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.308035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:51041 (local address 127.22.63.67:39435)
0806 01:07:56.308239 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:56.308247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:56.308867 (+ 620us) server_negotiation.cc:407] Connection header received
0806 01:07:56.309117 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:56.309123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:56.309215 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:56.309315 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:07:56.309703 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.310831 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.312474 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.313519 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.313667 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.314290 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:07:56.314397 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:07:56.314474 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:56.314661 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 01:07:56.314730 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:07:56.315276 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.308069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:39435 (local address 127.22.63.66:51041)
0806 01:07:56.308751 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0806 01:07:56.308768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:07:56.308936 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:07:56.309348 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:07:56.309352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:07:56.309362 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 01:07:56.309559 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:56.309567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.310976 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.310980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:56.312268 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:07:56.312283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.313838 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.313846 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:07:56.314053 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.314109 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 01:07:56.314520 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 01:07:56.314533 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 01:07:56.314631 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 01:07:56.314790 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":394,"threads_started":1}
I20260806 01:07:56.350282 1358 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:56.350591 1358 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:07:56.354287 1358 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 44b0da2b8d7245a68b62569269d04b1f: Sending DeleteTablet for 3 replicas of tablet 7f7625ec496d46bdbfc2cbe3e2093a81
I20260806 01:07:56.355150 1564 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:56 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:37414
I20260806 01:07:56.355423 2135 tablet_replica.cc:330] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: stopping tablet replica
I20260806 01:07:56.355476 1958 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:56 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:55214
I20260806 01:07:56.355615 2135 raft_consensus.cc:2229] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:56.355830 2135 raft_consensus.cc:2258] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:56.355832 2134 tablet_replica.cc:330] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: stopping tablet replica
I20260806 01:07:56.356025 2134 raft_consensus.cc:2229] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:07:56.356417 2134 raft_consensus.cc:2258] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:56.357169 2135 ts_tablet_manager.cc:1905] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:56.358071 2134 ts_tablet_manager.cc:1905] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:56.361979 1756 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f7625ec496d46bdbfc2cbe3e2093a81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:07:56 UTC) from {username='oryx', principal='oryx/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:38240
I20260806 01:07:56.362282 2133 tablet_replica.cc:330] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: stopping tablet replica
I20260806 01:07:56.362515 2133 raft_consensus.cc:2229] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:07:56.362731 2133 raft_consensus.cc:2258] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:07:56.363950 2133 ts_tablet_manager.cc:1905] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:07:56.364710 2135 ts_tablet_manager.cc:1918] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:56.364869 2135 log.cc:1196] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/7f7625ec496d46bdbfc2cbe3e2093a81
I20260806 01:07:56.365285 2135 ts_tablet_manager.cc:1939] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 7bc7c70276af41079fa30e7b455ca47e: Deleting consensus metadata
I20260806 01:07:56.366411 1336 catalog_manager.cc:4841] TS 7bc7c70276af41079fa30e7b455ca47e (127.22.63.65:42469): tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (table test-table [id=2f807bb785c04633a74e09308f6949b9]) successfully deleted
I20260806 01:07:56.366638 2134 ts_tablet_manager.cc:1918] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:56.367245 2134 log.cc:1196] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/7f7625ec496d46bdbfc2cbe3e2093a81
I20260806 01:07:56.367830 2134 ts_tablet_manager.cc:1939] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 54254e1c5b484ca7aaddcf1836d4e15d: Deleting consensus metadata
I20260806 01:07:56.369063 1335 catalog_manager.cc:4841] TS 54254e1c5b484ca7aaddcf1836d4e15d (127.22.63.67:39435): tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (table test-table [id=2f807bb785c04633a74e09308f6949b9]) successfully deleted
I20260806 01:07:56.369524 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 1463
I20260806 01:07:56.370604 2133 ts_tablet_manager.cc:1918] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 01:07:56.370781 2133 log.cc:1196] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/7f7625ec496d46bdbfc2cbe3e2093a81
I20260806 01:07:56.371286 2133 ts_tablet_manager.cc:1939] T 7f7625ec496d46bdbfc2cbe3e2093a81 P 2d1121839e62429ba2354609ac5dde20: Deleting consensus metadata
I20260806 01:07:56.376670 1337 catalog_manager.cc:4841] TS 2d1121839e62429ba2354609ac5dde20 (127.22.63.66:34283): tablet 7f7625ec496d46bdbfc2cbe3e2093a81 (table test-table [id=2f807bb785c04633a74e09308f6949b9]) successfully deleted
Aug 06 01:07:56 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.65@KRBTEST.COM
I20260806 01:07:56.402578 2240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.370524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.22.63.65:42469)
0806 01:07:56.384205 (+ 13681us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:56.384209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:56.385794 (+ 1585us) server_negotiation.cc:407] Connection header received
0806 01:07:56.385996 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:56.386002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:56.386084 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:56.386208 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:56.388253 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.389059 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.390333 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.390753 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.390891 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.395259 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:56.395278 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:56.395298 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:56.395340 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:56.399438 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.400267 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.400277 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.400283 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.400468 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.400883 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.400889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.400893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.401368 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:56.401715 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:56.401896 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 01:07:56.402148 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13553,"thread_start_us":13426,"threads_started":1}
I20260806 01:07:56.403414 1653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:56.502527 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 1463
I20260806 01:07:56.531304 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 1667
Aug 06 01:07:56 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.66@KRBTEST.COM
I20260806 01:07:56.547876 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.532189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.22.63.66:34283)
0806 01:07:56.532429 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:56.532436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:56.532516 (+ 80us) server_negotiation.cc:407] Connection header received
0806 01:07:56.532743 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:56.532748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:56.532829 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:56.532948 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:56.534186 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.534896 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.536288 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.536734 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.536886 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.541233 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:56.541250 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:56.541274 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:56.541318 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:56.544717 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.545492 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.545503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.545508 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.545665 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.546063 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.546069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.546073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.546481 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:56.546847 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:56.547191 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 01:07:56.547433 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:07:56.548595 1854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:56.702181 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 1667
I20260806 01:07:56.738338 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 1868
Aug 06 01:07:56 dist-test-slave-7f01 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.63.67@KRBTEST.COM
I20260806 01:07:56.755717 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:07:56.739217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55240 (local address 127.22.63.67:39435)
0806 01:07:56.739466 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:07:56.739471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:07:56.739502 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:07:56.739805 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:07:56.739811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:07:56.739949 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:07:56.740094 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:07:56.742169 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.742856 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.744882 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:07:56.745467 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:07:56.745599 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:07:56.750029 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:07:56.750043 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:07:56.750060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:07:56.750094 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:07:56.752944 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.753623 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.753638 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.753641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.753762 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:07:56.754141 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:07:56.754148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:07:56.754151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:07:56.754516 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:07:56.754816 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 01:07:56.755126 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 01:07:56.755359 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 01:07:56.756462 2054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:56.883402 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 1868
I20260806 01:07:56.914335 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 1304
I20260806 01:07:56.915642 1416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:07:57.034700 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 1304
2026-08-06T01:07:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7452 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:57 dist-test-slave-7f01 krb5kdc[2249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:57 dist-test-slave-7f01 krb5kdc[2249](info): set up 2 sockets
Aug 06 01:07:57 dist-test-slave-7f01 krb5kdc[2249](info): commencing operation
krb5kdc: starting...
W20260806 01:07:58.381202 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:07:58 dist-test-slave-7f01 krb5kdc[2249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:07:58.677181 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:35903
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:36431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:35903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:07:59.027818 2268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:07:59.028198 2268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:07:59.028358 2268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:07:59.035722 2268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:07:59.035898 2268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:07:59.036015 2268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:07:59.036116 2268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:07:59.036197 2268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:07:59.046241 2268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:35903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:35903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:07:59.047649 2268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:07:59.049014 2268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:07:59.061352 2275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:59.061430 2277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:07:59.061475 2281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:07:59.062690 2268 server_base.cc:1034] running on GCE node
I20260806 01:07:59.063549 2268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:07:59.065148 2268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:07:59.066390 2268 hybrid_clock.cc:648] HybridClock initialized: now 1785978479066351 us; error 50 us; skew 500 ppm
I20260806 01:07:59.066799 2268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T01:07:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2099 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:07:59 dist-test-slave-7f01 krb5kdc[2294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:07:59 dist-test-slave-7f01 krb5kdc[2294](info): set up 2 sockets
Aug 06 01:07:59 dist-test-slave-7f01 krb5kdc[2294](info): commencing operation
krb5kdc: starting...
W20260806 01:08:00.483297 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:08:00 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
W20260806 01:08:00.995174 22781 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.63.126: real 0.106s user 0.001s sys 0.040s
I20260806 01:08:00.997413 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:46399
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:46399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:01.532051 2318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:01.532429 2318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:01.532581 2318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:01.540124 2318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:08:01.540282 2318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:01.540388 2318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:01.540490 2318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:08:01.540566 2318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:08:01.550882 2318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:46399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:46399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:01.552402 2318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:01.553771 2318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:01.566044 2328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:01.565981 2331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:01.566505 2334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:01.567057 2318 server_base.cc:1034] running on GCE node
I20260806 01:08:01.568035 2318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:01.569542 2318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:01.570829 2318 hybrid_clock.cc:648] HybridClock initialized: now 1785978481570756 us; error 58 us; skew 500 ppm
I20260806 01:08:01.571214 2318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:01 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978481, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:01.574694 2318 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:08:01.576576 2318 webserver.cc:460] Webserver started at http://127.22.63.126:43213/ using document root <none> and password file <none>
I20260806 01:08:01.577374 2318 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:01.577536 2318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:01.577880 2318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:01.581331 2318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "290ac44f90a54422ba19a364ee2b4a0d"
format_stamp: "Formatted at 2026-08-06 01:08:01 on dist-test-slave-7f01"
server_key: "bdb0d8f2c819838b253fe5e9e6f8077b"
server_key_iv: "8729c7d63db89424265944dd0729bd97"
server_key_version: "encryptionkey@0"
I20260806 01:08:01.582187 2318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "290ac44f90a54422ba19a364ee2b4a0d"
format_stamp: "Formatted at 2026-08-06 01:08:01 on dist-test-slave-7f01"
server_key: "bdb0d8f2c819838b253fe5e9e6f8077b"
server_key_iv: "8729c7d63db89424265944dd0729bd97"
server_key_version: "encryptionkey@0"
I20260806 01:08:01.588177 2318 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 01:08:01.592183 2345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:01.593747 2318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:08:01.593973 2318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "290ac44f90a54422ba19a364ee2b4a0d"
format_stamp: "Formatted at 2026-08-06 01:08:01 on dist-test-slave-7f01"
server_key: "bdb0d8f2c819838b253fe5e9e6f8077b"
server_key_iv: "8729c7d63db89424265944dd0729bd97"
server_key_version: "encryptionkey@0"
I20260806 01:08:01.594201 2318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:01.608968 2318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:01.613868 2318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:01.614203 2318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:01.635105 2318 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:46399
I20260806 01:08:01.635128 2420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:46399 every 8 connection(s)
I20260806 01:08:01.636761 2318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:08:01.639237 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 2318
I20260806 01:08:01.639513 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:08:01.640419 22781 external_mini_cluster.cc:1428] Setting key 979af2d8e233a9a10f15cfc3ccd22d51
I20260806 01:08:01.641670 2422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:01.655498 2422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Bootstrap starting.
Aug 06 01:08:01 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:01.659225 2422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:01.660461 2422 log.cc:824] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:01.664057 2422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: No bootstrap required, opened a new log
I20260806 01:08:01.666703 2425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:01.642712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.22.63.126:46399)
0806 01:08:01.643669 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:01.643681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:01.643725 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:08:01.645453 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:01.645476 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:01.645894 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:01.646345 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:01.648096 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:01.649732 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:01.651371 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:01.652223 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:01.652491 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:01.657100 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:01.657110 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:01.657129 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:01.657182 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:01.660639 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:01.661880 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:01.661892 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:01.661903 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:01.662226 (+ 323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:01.662786 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:01.662792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:01.662798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:01.663592 (+ 794us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:01.664010 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:01.664737 (+ 727us) server_negotiation.cc:299] Negotiation successful
0806 01:08:01.665308 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":389,"threads_started":1}
I20260806 01:08:01.673014 2422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:01.673457 2422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:01.673580 2422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290ac44f90a54422ba19a364ee2b4a0d, State: Initialized, Role: FOLLOWER
I20260806 01:08:01.674247 2422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:01.674583 2422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:08:01.674750 2422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:08:01.674957 2422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:01.677345 2422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:01.677976 2422 leader_election.cc:302] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 290ac44f90a54422ba19a364ee2b4a0d; no voters:
I20260806 01:08:01.678689 2422 leader_election.cc:288] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:08:01.678812 2427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:08:01.679455 2427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 LEADER]: Becoming Leader. State: Replica: 290ac44f90a54422ba19a364ee2b4a0d, State: Running, Role: LEADER
I20260806 01:08:01.680147 2427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:01.681213 2422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:08:01.683607 2433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290ac44f90a54422ba19a364ee2b4a0d. Latest consensus state: current_term: 1 leader_uuid: "290ac44f90a54422ba19a364ee2b4a0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } } }
I20260806 01:08:01.684435 2433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:01.685132 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "290ac44f90a54422ba19a364ee2b4a0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } } }
I20260806 01:08:01.685477 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:01.687516 2446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:08:01.692919 2446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:08:01.705282 2446 catalog_manager.cc:1347] Generated new cluster ID: 9c30a13ec6e045fc97abacd3ad1d7de9
I20260806 01:08:01.705441 2446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:08:01.727609 2446 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:08:01.729251 2446 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:08:01.737179 2446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Generated new TSK 0
I20260806 01:08:01.738164 2446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:08:01.829000 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:02.241982 2464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:02.242303 2464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:02.242404 2464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:02.249990 2464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:02.250121 2464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:02.250239 2464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:08:02.260628 2464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:02.262051 2464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:02.263500 2464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:02.280242 2482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:02.280361 2476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:02.280341 2480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:02.280970 2464 server_base.cc:1034] running on GCE node
I20260806 01:08:02.281791 2464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:02.283381 2464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:02.284736 2464 hybrid_clock.cc:648] HybridClock initialized: now 1785978482284647 us; error 56 us; skew 500 ppm
I20260806 01:08:02.285369 2464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:02 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978482, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:02.290321 2464 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:08:02.292312 2464 webserver.cc:460] Webserver started at http://127.22.63.65:45019/ using document root <none> and password file <none>
I20260806 01:08:02.293136 2464 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:02.293303 2464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:02.293653 2464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:02.297017 2464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "e49fe65d1656448084096e1f7f7cc05e"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "f8a08b3ac098467a98add212b19d1fd6"
server_key_iv: "709d67d0718d16f158bcebf9b5ac28b2"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.298000 2464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "e49fe65d1656448084096e1f7f7cc05e"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "f8a08b3ac098467a98add212b19d1fd6"
server_key_iv: "709d67d0718d16f158bcebf9b5ac28b2"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.304595 2464 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 01:08:02.308673 2493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.310464 2464 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:08:02.310691 2464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "e49fe65d1656448084096e1f7f7cc05e"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "f8a08b3ac098467a98add212b19d1fd6"
server_key_iv: "709d67d0718d16f158bcebf9b5ac28b2"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.310945 2464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:02.367779 2464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:02.372803 2464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:02.373133 2464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:02.374198 2464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:02.375667 2464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:02.375788 2464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.375921 2464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:02.375991 2464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.413893 2464 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:43505
I20260806 01:08:02.414131 2692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:43505 every 8 connection(s)
I20260806 01:08:02.415500 2464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:08:02.420897 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 2464
I20260806 01:08:02.421178 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:08:02.421826 22781 external_mini_cluster.cc:1428] Setting key d28aa110eab26c50b287f8389bb735fc
Aug 06 01:08:02 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978482, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:02.440840 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:02.418559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:40833 (local address 127.22.63.126:46399)
0806 01:08:02.419091 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:02.419096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:02.420629 (+ 1533us) server_negotiation.cc:407] Connection header received
0806 01:08:02.422920 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:02.422927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:02.423057 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:02.423173 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:02.425111 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:02.426330 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:02.428151 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:02.428848 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:02.429002 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:02.433109 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:02.433119 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:02.433126 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:02.433177 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:02.436861 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:02.438083 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:02.438088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:02.438090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:02.438224 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:02.438804 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:02.438808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:02.438811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:02.439128 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:02.439265 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:02.440370 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0806 01:08:02.440566 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":285,"threads_started":1}
I20260806 01:08:02.442979 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:02.419176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:40833)
0806 01:08:02.420425 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:02.420480 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:02.422586 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:02.423463 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:02.423473 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:02.424284 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:02.424918 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:02.424936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:02.426546 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:02.426550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:02.427996 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:02.428010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:02.429090 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:02.429095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:02.429329 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:02.430148 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:02.430172 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:02.432818 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:02.437102 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:02.437117 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:02.437141 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:02.437901 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:02.438354 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:02.438362 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:02.438368 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:02.438640 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:02.439288 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:02.439300 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:02.439866 (+ 566us) client_negotiation.cc:769] Sending connection context
0806 01:08:02.440218 (+ 352us) client_negotiation.cc:240] Negotiation successful
0806 01:08:02.440867 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":545,"threads_started":1}
I20260806 01:08:02.444831 2693 heartbeater.cc:347] Connected to a master server at 127.22.63.126:46399
I20260806 01:08:02.445171 2693 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:02.445966 2693 heartbeater.cc:511] Master 127.22.63.126:46399 requested a full tablet report, sending...
I20260806 01:08:02.448307 2370 ts_manager.cc:194] Registered new tserver with Master: e49fe65d1656448084096e1f7f7cc05e (127.22.63.65:43505)
I20260806 01:08:02.449802 2370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:40833
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:08:02.491595 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:02.862130 2705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:02.862560 2705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:02.862736 2705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:02.870573 2705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:02.870718 2705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:02.870904 2705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:08:02.881954 2705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:02.883507 2705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:02.885059 2705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:02.900767 2715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:02.900815 2718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:02.900729 2716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:02.901988 2705 server_base.cc:1034] running on GCE node
I20260806 01:08:02.902675 2705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:02.903725 2705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:02.905056 2705 hybrid_clock.cc:648] HybridClock initialized: now 1785978482904980 us; error 58 us; skew 500 ppm
I20260806 01:08:02.905483 2705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:02 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978482, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:02.909209 2705 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:08:02.910838 2705 webserver.cc:460] Webserver started at http://127.22.63.66:34215/ using document root <none> and password file <none>
I20260806 01:08:02.911609 2705 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:02.911718 2705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:02.912125 2705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:02.916005 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "d722b44d66e4477ca97e8e05620d14f8"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "915c5250b500711f29e13d60de0d2d0c"
server_key_iv: "aea19b8f52a5ac590d3d3e02892f253d"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.916983 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "d722b44d66e4477ca97e8e05620d14f8"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "915c5250b500711f29e13d60de0d2d0c"
server_key_iv: "aea19b8f52a5ac590d3d3e02892f253d"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.924026 2705 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 01:08:02.929138 2732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.930868 2705 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:08:02.931341 2705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "d722b44d66e4477ca97e8e05620d14f8"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "915c5250b500711f29e13d60de0d2d0c"
server_key_iv: "aea19b8f52a5ac590d3d3e02892f253d"
server_key_version: "encryptionkey@0"
I20260806 01:08:02.931715 2705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:02.953024 2705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:02.958161 2705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:02.958503 2705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:02.959583 2705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:02.961050 2705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:02.961128 2705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.961272 2705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:02.961325 2705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:02.999820 2705 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:46825
I20260806 01:08:02.999922 2905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:46825 every 8 connection(s)
I20260806 01:08:03.001315 2705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:08:03.004161 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 2705
I20260806 01:08:03.004388 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:08:03.004983 22781 external_mini_cluster.cc:1428] Setting key bb76787a9f2a5b3503cb174af4270726
Aug 06 01:08:03 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978482, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:03.030100 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.004820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:37601 (local address 127.22.63.126:46399)
0806 01:08:03.005393 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:03.005397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:03.006643 (+ 1246us) server_negotiation.cc:407] Connection header received
0806 01:08:03.009499 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:03.009504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:03.009574 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:03.009677 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:03.012046 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.013094 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.016104 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.016692 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.016846 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.022440 (+ 5594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:03.022446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:03.022450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:03.022492 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:03.026122 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.027253 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.027259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.027261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.027401 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.028023 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.028027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.028029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.028442 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:03.028565 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:03.029596 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0806 01:08:03.029790 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":350,"threads_started":1}
I20260806 01:08:03.031925 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.005531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:37601)
0806 01:08:03.006466 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:03.006514 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:03.009142 (+ 2628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:03.010004 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:03.010019 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:03.010899 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:03.011820 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:03.011861 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.013400 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.013409 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:03.015768 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:03.015791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.017024 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.017034 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:03.017396 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.018292 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:03.018335 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:03.022089 (+ 3754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:03.026306 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:03.026320 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:03.026343 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:03.027076 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:03.027534 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:03.027541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:03.027548 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:03.027843 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:03.028594 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:03.028606 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:03.029160 (+ 554us) client_negotiation.cc:769] Sending connection context
0806 01:08:03.029502 (+ 342us) client_negotiation.cc:240] Negotiation successful
0806 01:08:03.030130 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":393,"threads_started":1}
I20260806 01:08:03.033732 2907 heartbeater.cc:347] Connected to a master server at 127.22.63.126:46399
I20260806 01:08:03.034071 2907 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:03.034905 2907 heartbeater.cc:511] Master 127.22.63.126:46399 requested a full tablet report, sending...
I20260806 01:08:03.036567 2370 ts_manager.cc:194] Registered new tserver with Master: d722b44d66e4477ca97e8e05620d14f8 (127.22.63.66:46825)
I20260806 01:08:03.037554 2370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:37601
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:08:03.077284 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:03.438289 2915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:03.438721 2915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:03.438902 2915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:03.446310 2915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:03.446501 2915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:03.446682 2915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:08:03.453462 2693 heartbeater.cc:503] Master 127.22.63.126:46399 was elected leader, sending a full tablet report...
I20260806 01:08:03.457720 2915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:03.459276 2915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:03.460757 2915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:03.476208 2927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:03.476336 2933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:03.476238 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:03.477514 2915 server_base.cc:1034] running on GCE node
I20260806 01:08:03.478528 2915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:03.479988 2915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:03.481238 2915 hybrid_clock.cc:648] HybridClock initialized: now 1785978483481182 us; error 59 us; skew 500 ppm
I20260806 01:08:03.481652 2915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:03 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978483, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:03.485968 2915 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:08:03.487641 2915 webserver.cc:460] Webserver started at http://127.22.63.67:36517/ using document root <none> and password file <none>
I20260806 01:08:03.488456 2915 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:03.488569 2915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:03.488852 2915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:03.492069 2915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "69c00063dd5c45dbaf08bd854fa12fc3"
format_stamp: "Formatted at 2026-08-06 01:08:03 on dist-test-slave-7f01"
server_key: "920466413b8f57cab0490de59180696c"
server_key_iv: "e268dfb9194cd61fc13dded62e454687"
server_key_version: "encryptionkey@0"
I20260806 01:08:03.492909 2915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "69c00063dd5c45dbaf08bd854fa12fc3"
format_stamp: "Formatted at 2026-08-06 01:08:03 on dist-test-slave-7f01"
server_key: "920466413b8f57cab0490de59180696c"
server_key_iv: "e268dfb9194cd61fc13dded62e454687"
server_key_version: "encryptionkey@0"
I20260806 01:08:03.499246 2915 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 01:08:03.502974 2943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:03.504482 2915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:08:03.504626 2915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "69c00063dd5c45dbaf08bd854fa12fc3"
format_stamp: "Formatted at 2026-08-06 01:08:03 on dist-test-slave-7f01"
server_key: "920466413b8f57cab0490de59180696c"
server_key_iv: "e268dfb9194cd61fc13dded62e454687"
server_key_version: "encryptionkey@0"
I20260806 01:08:03.504812 2915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:03.518599 2915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:03.523168 2915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:03.523437 2915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:03.524639 2915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:03.526053 2915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:03.526132 2915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:03.526221 2915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:03.526278 2915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:03.565106 2915 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:42763
I20260806 01:08:03.565261 3115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:42763 every 8 connection(s)
I20260806 01:08:03.566547 2915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:08:03.571601 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 2915
I20260806 01:08:03.571858 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:08:03.572484 22781 external_mini_cluster.cc:1428] Setting key b82e4c6b11a57de09a6327cfbbaa4346
Aug 06 01:08:03 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978483, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:03.590737 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.569646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:48619 (local address 127.22.63.126:46399)
0806 01:08:03.570343 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:03.570352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:03.571292 (+ 940us) server_negotiation.cc:407] Connection header received
0806 01:08:03.573916 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:03.573924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:03.574070 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:03.574210 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:03.576299 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.577201 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.578984 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.579495 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.579603 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.583678 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:03.583684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:03.583688 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:03.583727 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:03.587085 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.588112 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.588117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.588120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.588305 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.588852 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.588856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.588859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.589148 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:03.589318 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:03.590079 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 01:08:03.590317 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":278,"threads_started":1}
I20260806 01:08:03.591972 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.570074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:48619)
0806 01:08:03.570954 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:03.571042 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:03.573602 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:03.574554 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:03.574567 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:03.575300 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:03.576096 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:03.576127 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.577356 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.577360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:03.578821 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:03.578836 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.579719 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.579724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:03.580048 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.580762 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:03.580787 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:03.583410 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:03.587317 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:03.587330 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:03.587351 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:03.587970 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:03.588426 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:03.588429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:03.588432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:03.588721 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:03.589292 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:03.589301 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:03.589682 (+ 381us) client_negotiation.cc:769] Sending connection context
0806 01:08:03.589983 (+ 301us) client_negotiation.cc:240] Negotiation successful
0806 01:08:03.590584 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":365,"threads_started":1}
I20260806 01:08:03.593623 3117 heartbeater.cc:347] Connected to a master server at 127.22.63.126:46399
I20260806 01:08:03.593940 3117 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:03.594727 3117 heartbeater.cc:511] Master 127.22.63.126:46399 requested a full tablet report, sending...
I20260806 01:08:03.596686 2370 ts_manager.cc:194] Registered new tserver with Master: 69c00063dd5c45dbaf08bd854fa12fc3 (127.22.63.67:42763)
I20260806 01:08:03.597555 2370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:48619
I20260806 01:08:03.601315 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
Entry for principal oryx/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab.
WARNING: no policy specified for oryx/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
Entry for principal oryx/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab.
WARNING: no policy specified for oryx/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
Entry for principal oryx/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab.
I20260806 01:08:03.788568 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 2464
Aug 06 01:08:03 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:08:03.806981 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.789936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58820 (local address 127.22.63.65:43505)
0806 01:08:03.790722 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:03.790735 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:03.791147 (+ 412us) server_negotiation.cc:407] Connection header received
0806 01:08:03.791383 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:03.791391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:03.791610 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:03.791821 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:03.793223 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.794228 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.795542 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.796187 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.796328 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.800803 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:03.800816 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:03.800832 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:03.800876 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:03.803929 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.804694 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.804702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.804704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.804845 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.805235 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.805240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.805243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.805629 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:03.805959 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:03.806282 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:08:03.806577 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":398,"threads_started":1}
I20260806 01:08:03.808383 2688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:03.898334 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 2464
I20260806 01:08:03.923364 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 2705
Aug 06 01:08:03 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:08:03.940881 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:03.924333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42926 (local address 127.22.63.66:46825)
0806 01:08:03.924921 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:03.924928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:03.924976 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:08:03.925048 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:03.925055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:03.925261 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:03.925460 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:03.926802 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.928031 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.929317 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:03.929880 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:03.930024 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:03.934444 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:03.934459 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:03.934474 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:03.934511 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:03.937895 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.938630 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.938637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.938640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.938767 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:03.939158 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:03.939163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:03.939166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:03.939541 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:03.939892 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:03.940229 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:08:03.940520 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":323,"threads_started":1}
I20260806 01:08:03.942149 2901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:04.026656 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 2705
I20260806 01:08:04.049883 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 2915
Aug 06 01:08:04 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:08:04.067564 3137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:04.051017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.22.63.67:42763)
0806 01:08:04.051594 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:04.051601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:04.051631 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:08:04.051693 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:04.051699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:04.052001 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:04.052134 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:04.053546 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:04.054567 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:04.056023 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:04.056575 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:04.056673 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:04.061373 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:04.061386 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:04.061402 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:04.061439 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:04.064633 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:04.065419 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:04.065427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:04.065431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:04.065568 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:04.065940 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:04.065944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:04.065946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:04.066372 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:04.066614 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:04.066894 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 01:08:04.067126 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":285,"threads_started":1}
I20260806 01:08:04.068995 3108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:04.155483 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 2915
I20260806 01:08:04.179530 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 2318
I20260806 01:08:04.181021 2413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:04.289232 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 2318
I20260806 01:08:04.318931 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:46399
--webserver_interface=127.22.63.126
--webserver_port=43213
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:46399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:04.670961 3142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:04.671331 3142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:04.671510 3142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:04.678848 3142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:08:04.679000 3142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:04.679121 3142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:04.679219 3142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:08:04.679308 3142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:08:04.689935 3142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:46399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:46399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=43213
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:04.691325 3142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:04.692795 3142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:04.705308 3152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:04.705250 3154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:04.705531 3150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:04.706179 3142 server_base.cc:1034] running on GCE node
I20260806 01:08:04.707099 3142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:04.708701 3142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:04.709957 3142 hybrid_clock.cc:648] HybridClock initialized: now 1785978484709897 us; error 42 us; skew 500 ppm
I20260806 01:08:04.710386 3142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:04 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978484, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:04.713878 3142 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:08:04.715481 3142 webserver.cc:460] Webserver started at http://127.22.63.126:43213/ using document root <none> and password file <none>
I20260806 01:08:04.716311 3142 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:04.716495 3142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:04.722270 3142 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:08:04.725378 3165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:04.726936 3142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:04.727144 3142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "290ac44f90a54422ba19a364ee2b4a0d"
format_stamp: "Formatted at 2026-08-06 01:08:01 on dist-test-slave-7f01"
server_key: "bdb0d8f2c819838b253fe5e9e6f8077b"
server_key_iv: "8729c7d63db89424265944dd0729bd97"
server_key_version: "encryptionkey@0"
I20260806 01:08:04.728312 3142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:04.740495 3142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:04.744455 3142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:04.744745 3142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:04.763918 3142 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:46399
I20260806 01:08:04.763938 3243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:46399 every 8 connection(s)
I20260806 01:08:04.765460 3142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:08:04.769414 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 3142
I20260806 01:08:04.770328 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:43505
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:46399
--webserver_port=45019
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:08:04.772679 3246 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:08:04.776021 3246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Bootstrap starting.
I20260806 01:08:04.798390 3246 log.cc:824] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:04.807461 3246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:08:04.808100 3246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Bootstrap complete.
I20260806 01:08:04.817198 3246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:04.818063 3246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 290ac44f90a54422ba19a364ee2b4a0d, State: Initialized, Role: FOLLOWER
I20260806 01:08:04.818748 3246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:04.819011 3246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:08:04.819146 3246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:08:04.819334 3246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:08:04.823177 3246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:04.823704 3246 leader_election.cc:302] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 290ac44f90a54422ba19a364ee2b4a0d; no voters:
I20260806 01:08:04.824477 3246 leader_election.cc:288] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:08:04.824626 3250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:08:04.826150 3250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [term 2 LEADER]: Becoming Leader. State: Replica: 290ac44f90a54422ba19a364ee2b4a0d, State: Running, Role: LEADER
I20260806 01:08:04.826838 3246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:08:04.826756 3250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } }
I20260806 01:08:04.829550 3251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "290ac44f90a54422ba19a364ee2b4a0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } } }
I20260806 01:08:04.829989 3251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:04.831804 3252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 290ac44f90a54422ba19a364ee2b4a0d. Latest consensus state: current_term: 2 leader_uuid: "290ac44f90a54422ba19a364ee2b4a0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "290ac44f90a54422ba19a364ee2b4a0d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 46399 } } }
I20260806 01:08:04.832263 3252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:04.832398 3256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:08:04.839257 3256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:08:04.841223 3256 catalog_manager.cc:1259] Loaded cluster ID: 9c30a13ec6e045fc97abacd3ad1d7de9
I20260806 01:08:04.841442 3256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:08:04.846431 3256 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:08:04.848343 3256 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 290ac44f90a54422ba19a364ee2b4a0d: Loaded TSK: 0
I20260806 01:08:04.850068 3256 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:08:05.149946 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:05.150296 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:05.150456 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:05.158092 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:05.158252 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:05.158427 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:08:05.168644 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.65.keytab
--principal=oryx/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:43505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=45019
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:05.170102 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:05.171496 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:05.186793 3288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:05.186807 3290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:05.187686 3294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:05.188158 3248 server_base.cc:1034] running on GCE node
I20260806 01:08:05.188905 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:05.189927 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:05.191171 3248 hybrid_clock.cc:648] HybridClock initialized: now 1785978485191116 us; error 47 us; skew 500 ppm
I20260806 01:08:05.191512 3248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978485, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:05.194886 3248 init.cc:377] Logged in from keytab as oryx/127.22.63.65@KRBTEST.COM (short username oryx)
I20260806 01:08:05.196555 3248 webserver.cc:460] Webserver started at http://127.22.63.65:45019/ using document root <none> and password file <none>
I20260806 01:08:05.197291 3248 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:05.197443 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:05.203859 3248 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:08:05.206722 3304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.208252 3248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:05.208464 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "e49fe65d1656448084096e1f7f7cc05e"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "f8a08b3ac098467a98add212b19d1fd6"
server_key_iv: "709d67d0718d16f158bcebf9b5ac28b2"
server_key_version: "encryptionkey@0"
I20260806 01:08:05.209470 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:05.238125 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:05.242321 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:05.242628 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:05.243685 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:05.245239 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:05.245348 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.245505 3248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:05.245613 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.280963 3248 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:43505
I20260806 01:08:05.281059 3502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:43505 every 8 connection(s)
I20260806 01:08:05.282554 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:08:05.287936 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 3248
I20260806 01:08:05.288708 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:46825
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:46399
--webserver_port=34215
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:08:05.645268 3508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:05.645581 3508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:05.645728 3508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:05.653162 3508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:05.653285 3508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:05.653452 3508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:08:05.663570 3508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.66.keytab
--principal=oryx/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:46825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=34215
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:05.665036 3508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:05.666389 3508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:05.685570 3525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:05.685521 3522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:05.685552 3520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:05.686866 3508 server_base.cc:1034] running on GCE node
I20260806 01:08:05.688068 3508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:05.689849 3508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:05.691179 3508 hybrid_clock.cc:648] HybridClock initialized: now 1785978485691074 us; error 72 us; skew 500 ppm
I20260806 01:08:05.691768 3508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978485, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:05.700239 3508 init.cc:377] Logged in from keytab as oryx/127.22.63.66@KRBTEST.COM (short username oryx)
I20260806 01:08:05.702214 3508 webserver.cc:460] Webserver started at http://127.22.63.66:34215/ using document root <none> and password file <none>
I20260806 01:08:05.702992 3508 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:05.703163 3508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:05.709367 3508 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 01:08:05.712319 3539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.713972 3508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:08:05.714159 3508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "d722b44d66e4477ca97e8e05620d14f8"
format_stamp: "Formatted at 2026-08-06 01:08:02 on dist-test-slave-7f01"
server_key: "915c5250b500711f29e13d60de0d2d0c"
server_key_iv: "aea19b8f52a5ac590d3d3e02892f253d"
server_key_version: "encryptionkey@0"
I20260806 01:08:05.715230 3508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:05.744203 3508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:05.749275 3508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:05.749590 3508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:05.750751 3508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:05.752395 3508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:05.752490 3508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.752638 3508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:05.752717 3508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:05.801738 3508 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:46825
I20260806 01:08:05.801790 3716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:46825 every 8 connection(s)
I20260806 01:08:05.803231 3508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:08:05.811214 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 3508
I20260806 01:08:05.811932 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:42763
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:46399
--webserver_port=36517
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:05 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 01:08:06.179620 3722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:06.180018 3722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:06.180202 3722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:06.188011 3722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:06.188158 3722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:06.188352 3722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:08:06.198325 3722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/krb5kdc/oryx_127.22.63.67.keytab
--principal=oryx/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:42763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=36517
--tserver_master_addrs=127.22.63.126:46399
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:06.199730 3722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:06.201429 3722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:06.219707 3745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:06.220310 3744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:06.227427 3748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:06.228055 3722 server_base.cc:1034] running on GCE node
I20260806 01:08:06.228952 3722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:06.230072 3722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:06.231364 3722 hybrid_clock.cc:648] HybridClock initialized: now 1785978486231294 us; error 57 us; skew 500 ppm
I20260806 01:08:06.232038 3722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978486, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:06.236114 3722 init.cc:377] Logged in from keytab as oryx/127.22.63.67@KRBTEST.COM (short username oryx)
I20260806 01:08:06.241117 3722 webserver.cc:460] Webserver started at http://127.22.63.67:36517/ using document root <none> and password file <none>
I20260806 01:08:06.241904 3722 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:06.242077 3722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:06.247947 3722 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 01:08:06.252292 3757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:06.253840 3722 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260806 01:08:06.254024 3722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "69c00063dd5c45dbaf08bd854fa12fc3"
format_stamp: "Formatted at 2026-08-06 01:08:03 on dist-test-slave-7f01"
server_key: "920466413b8f57cab0490de59180696c"
server_key_iv: "e268dfb9194cd61fc13dded62e454687"
server_key_version: "encryptionkey@0"
I20260806 01:08:06.254994 3722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:06.283854 3722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:06.292934 3722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:06.293255 3722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:06.294458 3722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:06.296072 3722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:06.296200 3722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:06.296346 3722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:06.296439 3722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:06.351528 3722 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:42763
I20260806 01:08:06.351578 3932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:42763 every 8 connection(s)
I20260806 01:08:06.353107 3722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:08:06.355450 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 3722
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:06.443900 3942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:06.357374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.22.63.126:46399)
0806 01:08:06.408241 (+ 50867us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:06.408252 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:06.412989 (+ 4737us) server_negotiation.cc:407] Connection header received
0806 01:08:06.413084 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:06.413092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:06.413202 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:06.413329 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:06.416885 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.418826 (+ 1941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.426424 (+ 7598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.427212 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.427407 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.431906 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:06.431922 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:06.431945 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:06.432049 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:06.437548 (+ 5499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:06.438648 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:06.438661 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:06.438673 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:06.438922 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:06.439453 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:06.439460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:06.439464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:06.440218 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:06.440566 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:06.442127 (+ 1561us) server_negotiation.cc:299] Negotiation successful
0806 01:08:06.442670 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50671,"thread_start_us":54427,"threads_started":1}
W20260806 01:08:06.829255 3507 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.532s user 0.004s sys 0.011s
W20260806 01:08:06.830767 3505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:05.285849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:41531 (local address 127.22.63.126:46399)
0806 01:08:05.286906 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:05.286945 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:05.287817 (+ 872us) server_negotiation.cc:407] Connection header received
0806 01:08:05.289903 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:05.289916 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:05.290246 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:05.290619 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:05.292266 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.293506 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.295129 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.295851 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.296069 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.830446 (+1534377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:41531: BlockingRecv error: recv got EOF from 127.22.63.65:41531 (error 108)
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":293,"threads_started":1}
I20260806 01:08:06.831389 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:05.285986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:41531)
0806 01:08:05.287127 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:05.287191 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:05.289322 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:05.290768 (+ 1446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:05.290781 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:05.291449 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:05.292063 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:05.292081 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.293682 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.293692 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:05.294969 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:05.294981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.296096 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.296104 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:05.296380 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:05.297096 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:05.297121 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:06.830173 (+1533052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":531,"threads_started":1}
W20260806 01:08:06.832300 3503 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:46399 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.63.126:46399: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:06.842836 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:06.833087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:37055)
0806 01:08:06.833287 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:06.833307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:06.833427 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:06.833825 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:06.833830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:06.834217 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:06.834384 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.834393 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.835689 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.835694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.837184 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.837202 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.838080 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.838084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.838202 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.838859 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:06.838877 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:06.842522 (+ 3645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:08:06.842979 3505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:06.833379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:37055 (local address 127.22.63.126:46399)
0806 01:08:06.833549 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:06.833554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:06.833570 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:08:06.833628 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:06.833631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:06.833685 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:06.833790 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:06.834534 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.835538 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.837351 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.837780 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.837860 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.842725 (+ 4865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:37055: BlockingRecv error: recv got EOF from 127.22.63.65:37055 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:06 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:06.852911 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:06.843804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:55111)
0806 01:08:06.844136 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:06.844150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:06.844286 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:06.844629 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:06.844636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:06.844954 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:06.845098 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.845107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.846117 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.846130 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.847064 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.847074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.848085 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.848092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.848276 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.848976 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:06.849001 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:06.852612 (+ 3611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251}
W20260806 01:08:06.853107 3505 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:06.843896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:55111 (local address 127.22.63.126:46399)
0806 01:08:06.844039 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:06.844043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:06.844224 (+ 181us) server_negotiation.cc:407] Connection header received
0806 01:08:06.844406 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:06.844411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:06.844492 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:06.844591 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:06.845239 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.845973 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.847204 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.847764 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.847888 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.852804 (+ 4916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:55111: BlockingRecv error: recv got EOF from 127.22.63.65:55111 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 01:08:06.853477 3503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:08:07.278390 3721 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.455s user 0.003s sys 0.015s
W20260806 01:08:07.280264 3720 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:05.806324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:37441 (local address 127.22.63.126:46399)
0806 01:08:05.807015 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:05.807022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:05.808182 (+ 1160us) server_negotiation.cc:407] Connection header received
0806 01:08:05.812616 (+ 4434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:05.812627 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:05.812767 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:05.812910 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:05.815531 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.817414 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.820353 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.821373 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.821575 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.280021 (+1458446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:37441: BlockingRecv error: recv got EOF from 127.22.63.66:37441 (error 108)
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":337,"threads_started":1}
I20260806 01:08:07.281356 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:05.806638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:37441)
0806 01:08:05.807943 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:05.808015 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:05.811902 (+ 3887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:05.813279 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:05.813297 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:05.814307 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:05.815226 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:05.815258 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.817656 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.817664 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:05.820046 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:05.820084 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:05.821702 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:05.821712 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:05.822069 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:05.823094 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:05.823131 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.279842 (+1456711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":473,"threads_started":1}
W20260806 01:08:07.282848 3717 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:46399 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.63.126:46399: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:07.298237 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:07.283603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:44259)
0806 01:08:07.284294 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:07.284325 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:07.284586 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:07.285360 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:07.285369 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:07.286138 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:07.286460 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.286491 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.287635 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.287643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.290067 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.290093 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.291019 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.291029 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.291307 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.292243 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:07.292279 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.298002 (+ 5723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":1523,"threads_started":1}
W20260806 01:08:07.298768 3720 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:07.283618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:44259 (local address 127.22.63.126:46399)
0806 01:08:07.283840 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:07.283850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:07.284427 (+ 577us) server_negotiation.cc:407] Connection header received
0806 01:08:07.284785 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:07.284794 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:07.284907 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:07.285022 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:07.286663 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.287458 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.290309 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.290771 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.290884 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.298382 (+ 7498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:44259: BlockingRecv error: recv got EOF from 127.22.63.66:44259 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:07.309921 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:07.299013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:55273)
0806 01:08:07.299163 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:07.299183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:07.299295 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:07.300159 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:07.300162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:07.300461 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:07.300621 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.300629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.302659 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.302663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.303675 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.303689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.305072 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.305077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.305208 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.305866 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:07.305890 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.309698 (+ 3808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 01:08:07.310187 3717 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:08:07.310461 3720 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:07.299497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:55273 (local address 127.22.63.126:46399)
0806 01:08:07.299728 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:07.299762 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:07.299787 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:08:07.299863 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:07.299871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:07.299996 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:07.300140 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:07.300780 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.302423 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.303888 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.304876 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.305052 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.310059 (+ 5007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:55273: BlockingRecv error: recv got EOF from 127.22.63.66:55273 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 01:08:07.620995 3940 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.187s user 0.003s sys 0.014s
W20260806 01:08:07.622110 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:06.377920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:59083 (local address 127.22.63.126:46399)
0806 01:08:06.411752 (+ 33832us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:06.411762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:06.411794 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:08:06.422131 (+ 10337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:06.422140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:06.422264 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:06.422374 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:06.426507 (+ 4133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.427818 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.430912 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.431549 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.431659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.621912 (+1190253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:59083: BlockingRecv error: recv got EOF from 127.22.63.67:59083 (error 108)
Metrics: {"server-negotiator.queue_time_us":33643,"thread_start_us":30340,"threads_started":1}
I20260806 01:08:07.622690 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:06.377026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:59083)
0806 01:08:06.404028 (+ 27002us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:06.404108 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:06.419443 (+ 15335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:06.423260 (+ 3817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:06.423279 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:06.424477 (+ 1198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:06.425616 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.425649 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.428056 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.428061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.430706 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:06.430723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:06.432098 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:06.432107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:06.432490 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:06.433532 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:06.433567 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.621724 (+1188157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26461,"thread_start_us":32331,"threads_started":1}
W20260806 01:08:07.623551 3935 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:46399 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.63.126:46399: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:07.633308 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:07.624196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:43195)
0806 01:08:07.624407 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:07.624424 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:07.624527 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:07.624873 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:07.624876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:07.625234 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:07.625412 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.625421 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.626450 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.626454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.627552 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.627565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.628458 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.628464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.628587 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.629194 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:07.629216 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.633056 (+ 3840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:08:07.633507 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:07.624321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:43195 (local address 127.22.63.126:46399)
0806 01:08:07.624511 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:07.624516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:07.624533 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:08:07.624641 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:07.624645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:07.624707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:07.624809 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:07.625539 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.626282 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.627702 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.628249 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.628345 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.633260 (+ 4915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:43195: BlockingRecv error: recv got EOF from 127.22.63.67:43195 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:07.642499 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:07.634045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:36289)
0806 01:08:07.634230 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:07.634252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:07.634355 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:07.634627 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:07.634631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:07.634904 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:07.635056 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.635063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.636096 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.636100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.637181 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.637193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.637919 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.637923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.638053 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.638681 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:07.638699 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.642208 (+ 3509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:08:07.642822 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:07.634121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:36289 (local address 127.22.63.126:46399)
0806 01:08:07.634369 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:07.634373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:07.634394 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:08:07.634436 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:07.634439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:07.634510 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:07.634587 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:07.635173 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.635961 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.637308 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.637754 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.637852 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.642491 (+ 4639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:36289: BlockingRecv error: recv got EOF from 127.22.63.67:36289 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
W20260806 01:08:07.642825 3935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:07 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:07.864320 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:07.854203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:51823)
0806 01:08:07.854804 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:07.854822 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:07.854949 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:07.855265 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:07.855268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:07.855892 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:07.856096 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.856107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.857212 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.857216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.858188 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:07.858200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.859134 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.859140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:07.859271 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.860020 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:07.860039 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:07.863958 (+ 3919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":347,"threads_started":1}
W20260806 01:08:07.864650 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:07.854235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:51823 (local address 127.22.63.126:46399)
0806 01:08:07.854424 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:07.854429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:07.854940 (+ 511us) server_negotiation.cc:407] Connection header received
0806 01:08:07.855055 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:07.855059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:07.855128 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:07.855217 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:07.856232 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.857069 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.858335 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:07.858913 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:07.859019 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:07.864201 (+ 5182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:51823: BlockingRecv error: recv got EOF from 127.22.63.65:51823 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:08.320796 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:08.310866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:54683)
0806 01:08:08.311439 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:08.311456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:08.311579 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:08.311933 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:08.311936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:08.312359 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:08.312513 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.312521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.313750 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.313754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.314790 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.314811 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.315573 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.315578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.315713 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.316471 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:08.316498 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:08.320445 (+ 3947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":336,"threads_started":1}
W20260806 01:08:08.320930 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:08.310984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:54683 (local address 127.22.63.126:46399)
0806 01:08:08.311180 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:08.311184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:08.311533 (+ 349us) server_negotiation.cc:407] Connection header received
0806 01:08:08.311684 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:08.311688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:08.311767 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:08.311865 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:08.312689 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.313556 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.314939 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.315371 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.315480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.320672 (+ 5192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:54683: BlockingRecv error: recv got EOF from 127.22.63.66:54683 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:08.653859 3949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:08.643695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:39993)
0806 01:08:08.644372 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:08.644389 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:08.644523 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:08.645019 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:08.645025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:08.645548 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:08.645753 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.645767 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.647203 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.647208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.648230 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.648243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.649051 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.649055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.649190 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.649880 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:08.649898 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:08.653594 (+ 3696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":390,"threads_started":1}
W20260806 01:08:08.654085 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:08.643874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:39993 (local address 127.22.63.126:46399)
0806 01:08:08.644130 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:08.644135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:08.644539 (+ 404us) server_negotiation.cc:407] Connection header received
0806 01:08:08.644640 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:08.644647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:08.644749 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:08.644896 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:08.645930 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.647047 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.648397 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.648862 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.648964 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.653859 (+ 4895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:39993: BlockingRecv error: recv got EOF from 127.22.63.67:39993 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:08 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:08.875326 3950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:08.865464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:40229)
0806 01:08:08.866086 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:08.866103 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:08.866293 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:08.866735 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:08.866743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:08.867400 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:08.867626 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.867639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.868966 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.868971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.870017 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:08.870030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.870759 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.870763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:08.870888 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.871590 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:08.871608 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:08.875064 (+ 3456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":364,"threads_started":1}
W20260806 01:08:08.875557 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:08.865598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:40229 (local address 127.22.63.126:46399)
0806 01:08:08.865862 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:08.865871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:08.866252 (+ 381us) server_negotiation.cc:407] Connection header received
0806 01:08:08.866467 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:08.866473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:08.866548 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:08.866704 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:08.867833 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.868784 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.870181 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:08.870616 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:08.870718 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:08.875296 (+ 4578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:40229: BlockingRecv error: recv got EOF from 127.22.63.65:40229 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:09.330852 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:09.321793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:49235)
0806 01:08:09.322286 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:09.322304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:09.322434 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:09.322712 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:09.322714 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:09.323157 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:09.323346 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.323355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.324388 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.324392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.325313 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.325325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.325993 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.325998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.326123 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.326785 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:09.326803 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:09.330566 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
W20260806 01:08:09.331032 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:09.321945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:49235 (local address 127.22.63.126:46399)
0806 01:08:09.322206 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:09.322212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:09.322384 (+ 172us) server_negotiation.cc:407] Connection header received
0806 01:08:09.322523 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:09.322527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:09.322587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:09.322665 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:09.323474 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.324266 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.325443 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.325842 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.325952 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.330747 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:49235: BlockingRecv error: recv got EOF from 127.22.63.66:49235 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:09.665126 3954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:09.655140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:36817)
0806 01:08:09.655689 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:09.655708 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:09.655928 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:09.656338 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:09.656342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:09.656746 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:09.656972 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.656989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.658153 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.658189 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.659155 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.659168 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.660029 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.660033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.660167 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.660850 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:09.660868 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:09.664860 (+ 3992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":318,"threads_started":1}
W20260806 01:08:09.665315 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:09.655389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:36817 (local address 127.22.63.126:46399)
0806 01:08:09.655618 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:09.655625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:09.655819 (+ 194us) server_negotiation.cc:407] Connection header received
0806 01:08:09.656080 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:09.656087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:09.656190 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:09.656276 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:09.657167 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.657981 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.659345 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.659872 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.659973 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.665076 (+ 5103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:36817: BlockingRecv error: recv got EOF from 127.22.63.67:36817 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:09 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:09.890341 3957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:09.876426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:41427)
0806 01:08:09.877055 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:09.877078 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:09.877339 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:09.877749 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:09.877754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:09.878180 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:09.878371 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.878380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.879895 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.879899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.881690 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:09.881706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.882874 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.882880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:09.883063 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.883823 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:09.883846 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:09.889980 (+ 6134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":278,"threads_started":1}
W20260806 01:08:09.890494 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:09.876773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:41427 (local address 127.22.63.126:46399)
0806 01:08:09.877055 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:09.877062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:09.877186 (+ 124us) server_negotiation.cc:407] Connection header received
0806 01:08:09.877520 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:09.877526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:09.877601 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:09.877709 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:09.878582 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.879680 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.881856 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:09.882536 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:09.882628 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:09.890245 (+ 7617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:41427: BlockingRecv error: recv got EOF from 127.22.63.65:41427 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:10.341450 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:10.332312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:40029)
0806 01:08:10.333088 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:10.333110 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:10.333237 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:10.333586 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:10.333589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:10.333951 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:10.334129 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.334137 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.335207 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.335210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.336155 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.336168 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.336844 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.336848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.336967 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.337559 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:10.337591 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:10.341173 (+ 3582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":278,"threads_started":1}
W20260806 01:08:10.341645 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:10.332501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:40029 (local address 127.22.63.126:46399)
0806 01:08:10.332753 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:10.332760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:10.333201 (+ 441us) server_negotiation.cc:407] Connection header received
0806 01:08:10.333358 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:10.333364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:10.333447 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:10.333556 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:10.334284 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.335075 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.336289 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.336709 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.336823 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.341391 (+ 4568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:40029: BlockingRecv error: recv got EOF from 127.22.63.66:40029 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:10.677203 3960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:10.666333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:39525)
0806 01:08:10.667166 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:10.667189 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:10.667364 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:10.667870 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:10.667879 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:10.668431 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:10.668676 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.668700 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.670180 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.670187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.671185 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.671197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.672107 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.672111 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.672232 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.672861 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:10.672879 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:10.676838 (+ 3959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":368,"threads_started":1}
W20260806 01:08:10.677317 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:10.666658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:39525 (local address 127.22.63.126:46399)
0806 01:08:10.666878 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:10.666887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:10.667293 (+ 406us) server_negotiation.cc:407] Connection header received
0806 01:08:10.667554 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:10.667564 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:10.667648 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:10.667769 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:10.668850 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.670032 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.671369 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.671832 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.671905 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.677058 (+ 5153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:39525: BlockingRecv error: recv got EOF from 127.22.63.67:39525 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:10 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:10.904394 3961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:10.891578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:59723)
0806 01:08:10.892159 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:10.892176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:10.892296 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:10.892627 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:10.892631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:10.893053 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:10.893297 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.893311 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.894683 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.894689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.895922 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:10.895936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.896773 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.896778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:10.896938 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.900364 (+ 3426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:10.900384 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:10.904109 (+ 3725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":331,"threads_started":1}
W20260806 01:08:10.904757 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:10.891640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:59723 (local address 127.22.63.126:46399)
0806 01:08:10.891879 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:10.891884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:10.892257 (+ 373us) server_negotiation.cc:407] Connection header received
0806 01:08:10.892404 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:10.892407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:10.892469 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:10.892567 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:10.893502 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.894508 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.896092 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:10.896606 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:10.896727 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:10.904418 (+ 7691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:59723: BlockingRecv error: recv got EOF from 127.22.63.65:59723 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:11.352468 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:11.342629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:40291)
0806 01:08:11.343149 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:11.343168 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:11.343290 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:11.343613 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:11.343616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:11.344044 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:11.344222 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.344230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.345344 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.345348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.346388 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.346402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.347221 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.347225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.347355 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.348017 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:11.348034 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:11.352091 (+ 4057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":287,"threads_started":1}
W20260806 01:08:11.352542 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:11.342681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:40291 (local address 127.22.63.126:46399)
0806 01:08:11.342928 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:11.342938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:11.343255 (+ 317us) server_negotiation.cc:407] Connection header received
0806 01:08:11.343392 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:11.343401 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:11.343472 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:11.343557 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:11.344361 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.345187 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.346548 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.347059 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.347166 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.352293 (+ 5127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:40291: BlockingRecv error: recv got EOF from 127.22.63.66:40291 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:11.689319 3963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:11.679414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:45565)
0806 01:08:11.680000 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:11.680017 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:11.680135 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:11.680489 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:11.680492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:11.680909 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:11.681079 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.681087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.682287 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.682291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.683536 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.683551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.684566 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.684571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.684701 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.685358 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:11.685389 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:11.688956 (+ 3567us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":285,"threads_started":1}
W20260806 01:08:11.689440 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:11.679558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:45565 (local address 127.22.63.126:46399)
0806 01:08:11.679861 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:11.679868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:11.680107 (+ 239us) server_negotiation.cc:407] Connection header received
0806 01:08:11.680245 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:11.680248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:11.680339 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:11.680427 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:11.681272 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.682107 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.683706 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.684416 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.684526 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.689172 (+ 4646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:45565: BlockingRecv error: recv got EOF from 127.22.63.67:45565 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:11 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:11.914601 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:11.905477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:53551)
0806 01:08:11.906017 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:11.906033 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:11.906149 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:11.906423 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:11.906427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:11.906838 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:11.907012 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.907021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.908062 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.908066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.909008 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:11.909025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.909638 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.909642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:11.909765 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.910384 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:11.910401 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:11.914223 (+ 3822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":275,"threads_started":1}
W20260806 01:08:11.914731 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:11.905524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:53551 (local address 127.22.63.126:46399)
0806 01:08:11.905704 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:11.905709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:11.906106 (+ 397us) server_negotiation.cc:407] Connection header received
0806 01:08:11.906230 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:11.906233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:11.906294 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:11.906374 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:11.907132 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.907933 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.909133 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:11.909512 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:11.909604 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:11.914451 (+ 4847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:53551: BlockingRecv error: recv got EOF from 127.22.63.65:53551 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:12.363179 3966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:12.353697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:35899)
0806 01:08:12.354251 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:12.354268 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:12.354435 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:12.354789 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:12.354794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:12.355260 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:12.355416 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.355425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.356869 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.356878 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.358055 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.358069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.358839 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.358843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.358964 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.359557 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:12.359575 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:12.362941 (+ 3366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
W20260806 01:08:12.363389 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:12.353958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:35899 (local address 127.22.63.126:46399)
0806 01:08:12.354165 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:12.354171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:12.354358 (+ 187us) server_negotiation.cc:407] Connection header received
0806 01:08:12.354566 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:12.354571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:12.354646 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:12.354757 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:12.355586 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.356713 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.358184 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.358711 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.358808 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.363138 (+ 4330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:35899: BlockingRecv error: recv got EOF from 127.22.63.66:35899 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:12.699935 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:12.690485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:48597)
0806 01:08:12.691018 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:12.691036 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:12.691161 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:12.691454 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:12.691457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:12.692014 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:12.692177 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.692186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.693331 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.693336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.694342 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.694354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.695129 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.695138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.695273 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.695986 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:12.696010 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:12.699615 (+ 3605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":293,"threads_started":1}
W20260806 01:08:12.700099 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:12.690560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:48597 (local address 127.22.63.126:46399)
0806 01:08:12.690811 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:12.690819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:12.691126 (+ 307us) server_negotiation.cc:407] Connection header received
0806 01:08:12.691249 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:12.691252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:12.691322 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:12.691407 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:12.692341 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.693176 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.694497 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.694943 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.695045 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.699869 (+ 4824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:48597: BlockingRecv error: recv got EOF from 127.22.63.67:48597 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:12 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:12.925040 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:12.915603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:52515)
0806 01:08:12.916213 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:12.916230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:12.916350 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:12.916635 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:12.916638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:12.917067 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:12.917222 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.917230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.918372 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.918377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.919407 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:12.919421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.920385 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.920391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:12.920536 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.921131 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:12.921149 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:12.924788 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":341,"threads_started":1}
W20260806 01:08:12.925266 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:12.915688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:52515 (local address 127.22.63.126:46399)
0806 01:08:12.915885 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:12.915889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:12.916322 (+ 433us) server_negotiation.cc:407] Connection header received
0806 01:08:12.916451 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:12.916454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:12.916518 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:12.916591 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:12.917379 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.918216 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.919566 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:12.920194 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:12.920310 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:12.925014 (+ 4704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:52515: BlockingRecv error: recv got EOF from 127.22.63.65:52515 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:13.375164 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:13.364268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:49387)
0806 01:08:13.364851 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:13.364870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:13.364991 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:13.365388 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:13.365392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:13.365837 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:13.366013 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.366022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.367015 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.367018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.368165 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.368183 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.369344 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.369348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.369478 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.370166 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:13.370186 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:13.374912 (+ 4726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":316,"threads_started":1}
W20260806 01:08:13.375669 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:13.364301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:49387 (local address 127.22.63.126:46399)
0806 01:08:13.364501 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:13.364506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:13.365037 (+ 531us) server_negotiation.cc:407] Connection header received
0806 01:08:13.365102 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:13.365106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:13.365217 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:13.365305 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:13.366147 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.366880 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.368355 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.369153 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.369299 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.375434 (+ 6135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:49387: BlockingRecv error: recv got EOF from 127.22.63.66:49387 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:13.710512 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:13.701009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:39461)
0806 01:08:13.701596 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:13.701629 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:13.701799 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:13.702174 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:13.702181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:13.702739 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:13.702909 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.702919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.704195 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.704200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.705124 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.705136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.705858 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.705863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.706056 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.706660 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:13.706678 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:13.710181 (+ 3503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":320,"threads_started":1}
W20260806 01:08:13.710668 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:13.701209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:39461 (local address 127.22.63.126:46399)
0806 01:08:13.701457 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:13.701463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:13.701717 (+ 254us) server_negotiation.cc:407] Connection header received
0806 01:08:13.701939 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:13.701944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:13.702022 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:13.702134 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:13.703063 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.704045 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.705274 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.705697 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.705831 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.710397 (+ 4566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:39461: BlockingRecv error: recv got EOF from 127.22.63.67:39461 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:13 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:13.935868 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:13.926060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:45283)
0806 01:08:13.926733 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:13.926750 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:13.926929 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:13.927297 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:13.927302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:13.927862 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:13.928057 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.928070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.929125 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.929132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.930098 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:13.930110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.930852 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.930860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:13.931046 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.931678 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:13.931696 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:13.935451 (+ 3755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":276,"threads_started":1}
W20260806 01:08:13.936245 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:13.926139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:45283 (local address 127.22.63.126:46399)
0806 01:08:13.926355 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:13.926364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:13.926879 (+ 515us) server_negotiation.cc:407] Connection header received
0806 01:08:13.927068 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:13.927074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:13.927148 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:13.927256 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:13.928216 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.928977 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.930244 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:13.930679 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:13.930815 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:13.935997 (+ 5182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:45283: BlockingRecv error: recv got EOF from 127.22.63.65:45283 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:14.385949 3972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:14.376435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:56329)
0806 01:08:14.377277 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:14.377295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:14.377457 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:14.377858 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:14.377862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:14.378484 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:14.378680 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.378689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.379696 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.379699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.380662 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.380674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.381366 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.381370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.381481 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.382091 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:14.382108 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:14.385682 (+ 3574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":504,"threads_started":1}
W20260806 01:08:14.386173 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:14.376544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:56329 (local address 127.22.63.126:46399)
0806 01:08:14.376800 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:14.376810 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:14.377365 (+ 555us) server_negotiation.cc:407] Connection header received
0806 01:08:14.377586 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:14.377590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:14.377652 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:14.377785 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:14.378820 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.379578 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.380793 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.381232 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.381328 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.385911 (+ 4583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:56329: BlockingRecv error: recv got EOF from 127.22.63.66:56329 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:14.721481 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:14.711629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:52847)
0806 01:08:14.712346 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:14.712365 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:14.712491 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:14.712860 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:14.712864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:14.713287 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:14.713462 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.713470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.714642 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.714646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.715631 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.715643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.716372 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.716377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.716501 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.717162 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:14.717182 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:14.721210 (+ 4028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":326,"threads_started":1}
W20260806 01:08:14.721639 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:14.711755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:52847 (local address 127.22.63.126:46399)
0806 01:08:14.711947 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:14.711952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:14.712451 (+ 499us) server_negotiation.cc:407] Connection header received
0806 01:08:14.712595 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:14.712599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:14.712662 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:14.712776 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:14.713597 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.714498 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.715789 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.716205 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.716301 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.721408 (+ 5107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:52847: BlockingRecv error: recv got EOF from 127.22.63.67:52847 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:14 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:14.946913 3976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:14.937322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:54515)
0806 01:08:14.937938 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:14.937955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:14.938094 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:14.938400 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:14.938403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:14.938826 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:14.939036 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.939046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.940251 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.940255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.941464 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:14.941477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.942322 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.942326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:14.942443 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.943134 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:14.943152 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:14.946653 (+ 3501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":318,"threads_started":1}
W20260806 01:08:14.947167 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:14.937481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54515 (local address 127.22.63.126:46399)
0806 01:08:14.937667 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:14.937671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:14.938050 (+ 379us) server_negotiation.cc:407] Connection header received
0806 01:08:14.938198 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:14.938202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:14.938263 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:14.938346 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:14.939201 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.940072 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.941615 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:14.942151 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:14.942250 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:14.946908 (+ 4658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:54515: BlockingRecv error: recv got EOF from 127.22.63.65:54515 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:15.396895 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:15.387010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:46567)
0806 01:08:15.387630 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:15.387648 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:15.387847 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:15.388233 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:15.388238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:15.388711 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:15.388881 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.388890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.390171 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.390177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.391127 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.391140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.391952 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.391959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.392127 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.392797 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:15.392815 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:15.396652 (+ 3837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":318,"threads_started":1}
W20260806 01:08:15.397159 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:15.387096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:46567 (local address 127.22.63.126:46399)
0806 01:08:15.387408 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:15.387415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:15.387768 (+ 353us) server_negotiation.cc:407] Connection header received
0806 01:08:15.387981 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:15.387989 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:15.388078 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:15.388190 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:15.389080 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.390006 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.391278 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.391710 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.391904 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.396883 (+ 4979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:46567: BlockingRecv error: recv got EOF from 127.22.63.66:46567 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:15.732414 3978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:15.722539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:49251)
0806 01:08:15.723184 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:15.723203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:15.723354 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:15.723844 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:15.723849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:15.724401 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:15.724632 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.724643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.725890 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.725894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.726865 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.726878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.727703 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.727707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.727936 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.728584 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:15.728602 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:15.732145 (+ 3543us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":371,"threads_started":1}
W20260806 01:08:15.732538 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:15.722679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:49251 (local address 127.22.63.126:46399)
0806 01:08:15.722861 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:15.722865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:15.723317 (+ 452us) server_negotiation.cc:407] Connection header received
0806 01:08:15.723491 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:15.723495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:15.723600 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:15.723721 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:15.724842 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.725710 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.727010 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.727541 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.727646 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.732315 (+ 4669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:49251: BlockingRecv error: recv got EOF from 127.22.63.67:49251 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:15 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:15.961489 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:15.947958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:54309)
0806 01:08:15.948517 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:15.948534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:15.948654 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:15.948968 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:15.948971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:15.949405 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:15.949559 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.949567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.950569 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.950573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.951553 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:15.951565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.952449 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.952454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:15.952592 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.956951 (+ 4359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:15.956978 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:15.961158 (+ 4180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":289,"threads_started":1}
W20260806 01:08:15.961613 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:15.948072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54309 (local address 127.22.63.126:46399)
0806 01:08:15.948279 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:15.948283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:15.948640 (+ 357us) server_negotiation.cc:407] Connection header received
0806 01:08:15.948741 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:15.948744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:15.948808 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:15.948900 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:15.949687 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.950432 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.951691 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:15.952248 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:15.952374 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:15.961352 (+ 8978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:54309: BlockingRecv error: recv got EOF from 127.22.63.65:54309 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:16.407299 3980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:16.398061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:60839)
0806 01:08:16.398601 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:16.398620 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:16.398739 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:16.399019 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:16.399022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:16.399461 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:16.399615 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.399623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.400734 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.400738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.401704 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.401718 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.402393 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.402397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.402523 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.403260 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:16.403286 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:16.407076 (+ 3790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":298,"threads_started":1}
W20260806 01:08:16.407548 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:16.398116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:60839 (local address 127.22.63.126:46399)
0806 01:08:16.398358 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:16.398367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:16.398708 (+ 341us) server_negotiation.cc:407] Connection header received
0806 01:08:16.398826 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:16.398830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:16.398894 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:16.398974 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:16.399778 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.400575 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.401845 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.402248 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.402347 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.407264 (+ 4917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:60839: BlockingRecv error: recv got EOF from 127.22.63.66:60839 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:16.743109 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:16.733512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:41545)
0806 01:08:16.734045 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:16.734062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:16.734241 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:16.734581 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:16.734586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:16.735082 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:16.735230 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.735240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.736389 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.736398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.737363 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.737375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.738108 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.738114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.738305 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.738962 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:16.738979 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:16.742876 (+ 3897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":287,"threads_started":1}
W20260806 01:08:16.743294 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:16.733560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:41545 (local address 127.22.63.126:46399)
0806 01:08:16.733849 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:16.733856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:16.734158 (+ 302us) server_negotiation.cc:407] Connection header received
0806 01:08:16.734379 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:16.734383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:16.734444 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:16.734549 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:16.735422 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.736245 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.737510 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.737949 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.738083 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.743075 (+ 4992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:41545: BlockingRecv error: recv got EOF from 127.22.63.67:41545 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:16 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:16.972271 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:16.962678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:34821)
0806 01:08:16.963277 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:16.963294 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:16.963418 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:16.963732 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:16.963754 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:16.964214 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:16.964372 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.964382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.965495 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.965499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.966509 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:16.966521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.967353 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.967357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:16.967474 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.968106 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:16.968124 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:16.971679 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":341,"threads_started":1}
W20260806 01:08:16.972461 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:16.962790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:34821 (local address 127.22.63.126:46399)
0806 01:08:16.963057 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:16.963062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:16.963406 (+ 344us) server_negotiation.cc:407] Connection header received
0806 01:08:16.963528 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:16.963532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:16.963598 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:16.963685 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:16.964537 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.965341 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.966658 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:16.967185 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:16.967299 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:16.972175 (+ 4876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:34821: BlockingRecv error: recv got EOF from 127.22.63.65:34821 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:17.417836 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:17.408400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:53077)
0806 01:08:17.409170 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:17.409188 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:17.409357 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:17.409761 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:17.409764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:17.410301 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:17.410535 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.410551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.411546 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.411549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.412652 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.412665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.413473 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.413477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.413598 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.414253 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:17.414271 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:17.417596 (+ 3325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":477,"threads_started":1}
W20260806 01:08:17.418043 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:17.408643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:53077 (local address 127.22.63.126:46399)
0806 01:08:17.408946 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:17.408955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:17.409285 (+ 330us) server_negotiation.cc:407] Connection header received
0806 01:08:17.409490 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:17.409495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:17.409611 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:17.409718 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:17.410700 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.411422 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.412794 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.413336 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.413433 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.417808 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:53077: BlockingRecv error: recv got EOF from 127.22.63.66:53077 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:17.753787 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:17.744113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:34785)
0806 01:08:17.744702 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:17.744720 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:17.744848 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:17.745162 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:17.745165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:17.745544 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:17.745718 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.745727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.746846 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.746850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.748049 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.748063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.749105 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.749112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.749245 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.750014 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:17.750033 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:17.753539 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":342,"threads_started":1}
W20260806 01:08:17.754068 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:17.744349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:34785 (local address 127.22.63.126:46399)
0806 01:08:17.744572 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:17.744577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:17.744795 (+ 218us) server_negotiation.cc:407] Connection header received
0806 01:08:17.744948 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:17.744952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:17.745026 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:17.745112 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:17.745907 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.746682 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.748297 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.748937 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.749044 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.753776 (+ 4732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:34785: BlockingRecv error: recv got EOF from 127.22.63.67:34785 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:17 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:17.983198 3987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:17.973552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:54979)
0806 01:08:17.974157 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:17.974181 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:17.974375 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:17.974821 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:17.974826 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:17.975395 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:17.975583 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.975592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.976795 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.976799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.977749 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:17.977761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.978495 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.978499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:17.978627 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.979265 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:17.979285 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:17.982956 (+ 3671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":314,"threads_started":1}
W20260806 01:08:17.983559 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:17.973608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54979 (local address 127.22.63.126:46399)
0806 01:08:17.973835 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:17.973841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:17.974309 (+ 468us) server_negotiation.cc:407] Connection header received
0806 01:08:17.974522 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:17.974528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:17.974627 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:17.974751 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:17.975729 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.976641 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.977894 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:17.978350 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:17.978448 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:17.983166 (+ 4718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:54979: BlockingRecv error: recv got EOF from 127.22.63.65:54979 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:18.428748 3988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:18.418906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:59019)
0806 01:08:18.419505 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:18.419538 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:18.419656 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:18.420037 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:18.420041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:18.420472 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:18.420724 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.420734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.421993 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.421999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.423216 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.423231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.423972 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.423977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.424094 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.424763 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:18.424781 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:18.428518 (+ 3737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":343,"threads_started":1}
W20260806 01:08:18.428977 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:18.419036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59019 (local address 127.22.63.126:46399)
0806 01:08:18.419235 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:18.419239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:18.419599 (+ 360us) server_negotiation.cc:407] Connection header received
0806 01:08:18.419808 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:18.419812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:18.419875 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:18.419985 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:18.420856 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.421836 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.423354 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.423836 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.423944 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.428710 (+ 4766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:59019: BlockingRecv error: recv got EOF from 127.22.63.66:59019 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:18.764243 3989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:18.754801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:60607)
0806 01:08:18.755333 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:18.755350 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:18.755472 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:18.755813 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:18.755819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:18.756262 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:18.756465 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.756474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.757539 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.757543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.758532 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.758549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.759371 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.759375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.759495 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.760193 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:18.760212 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:18.764015 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":285,"threads_started":1}
W20260806 01:08:18.764446 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:18.754962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:60607 (local address 127.22.63.126:46399)
0806 01:08:18.755186 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:18.755192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:18.755459 (+ 267us) server_negotiation.cc:407] Connection header received
0806 01:08:18.755557 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:18.755560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:18.755625 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:18.755711 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:18.756603 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.757409 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.758756 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.759228 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.759325 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.764204 (+ 4879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:60607: BlockingRecv error: recv got EOF from 127.22.63.67:60607 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:18 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:18.994123 3993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:18.984299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:53081)
0806 01:08:18.984939 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:18.984958 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:18.985133 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:18.985526 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:18.985534 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:18.985963 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:18.986153 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.986163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.987275 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.987282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.988256 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:18.988276 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.989034 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.989040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:18.989243 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.989891 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:18.989910 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:18.993742 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":343,"threads_started":1}
W20260806 01:08:18.994258 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:18.984388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:53081 (local address 127.22.63.126:46399)
0806 01:08:18.984688 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:18.984696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:18.985059 (+ 363us) server_negotiation.cc:407] Connection header received
0806 01:08:18.985295 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:18.985299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:18.985366 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:18.985478 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:18.986311 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.987114 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.988414 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:18.988879 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:18.989019 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:18.993990 (+ 4971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:53081: BlockingRecv error: recv got EOF from 127.22.63.65:53081 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:19.439790 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:19.429750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:44931)
0806 01:08:19.430321 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:19.430341 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:19.430515 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:19.430882 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:19.430887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:19.431310 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:19.431496 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.431504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.432834 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.432857 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:19.434213 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.434226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.434968 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.434972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:19.435109 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:19.435762 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:19.435781 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:19.439477 (+ 3696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":305,"threads_started":1}
W20260806 01:08:19.440045 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:19.429836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:44931 (local address 127.22.63.126:46399)
0806 01:08:19.430090 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:19.430099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:19.430438 (+ 339us) server_negotiation.cc:407] Connection header received
0806 01:08:19.430658 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:19.430662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:19.430726 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:19.430834 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:19.431649 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.432680 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.434367 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.434813 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.434917 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:19.439696 (+ 4779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:44931: BlockingRecv error: recv got EOF from 127.22.63.66:44931 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:19.776530 3999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:19.765293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:49509)
0806 01:08:19.765879 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:19.765902 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:19.766050 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:19.766706 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:19.766710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:19.767291 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:19.767542 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.767552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.768760 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.768766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:19.770285 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.770302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.771098 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.771104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:19.771299 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:19.772041 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:19.772064 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:19.776167 (+ 4103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":608,"threads_started":1}
W20260806 01:08:19.776646 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:19.765334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:49509 (local address 127.22.63.126:46399)
0806 01:08:19.765533 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:19.765540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:19.765975 (+ 435us) server_negotiation.cc:407] Connection header received
0806 01:08:19.766204 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:19.766216 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:19.766321 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:19.766419 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:19.767684 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.768568 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.770450 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.770890 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.770985 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:19.776399 (+ 5414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:49509: BlockingRecv error: recv got EOF from 127.22.63.67:49509 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:19 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:20.005111 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:19.995375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:39929)
0806 01:08:19.996005 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:19.996023 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:19.996147 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:19.996478 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:19.996482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:19.996888 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:19.997067 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.997075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.998372 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.998378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:19.999598 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:19.999611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.000341 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.000345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:20.000463 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.001049 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:20.001066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:20.004677 (+ 3611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":375,"threads_started":1}
W20260806 01:08:20.005224 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:19.995481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:39929 (local address 127.22.63.126:46399)
0806 01:08:19.995708 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:19.995712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:19.996105 (+ 393us) server_negotiation.cc:407] Connection header received
0806 01:08:19.996251 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:19.996255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:19.996328 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:19.996417 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:19.997203 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:19.998221 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:19.999756 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.000211 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.000315 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.004914 (+ 4599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:39929: BlockingRecv error: recv got EOF from 127.22.63.65:39929 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 01:08:20 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:20 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.66@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:20.450347 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:20.440807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.66:36125)
0806 01:08:20.441580 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:20.441598 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:20.441711 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:20.442027 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:20.442030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:20.442414 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:20.442579 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:20.442587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.443557 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.443560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:20.445174 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:20.445188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.445908 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.445911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:20.446049 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.446637 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:20.446655 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:20.450114 (+ 3459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":534,"threads_started":1}
W20260806 01:08:20.450594 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:20.441038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:36125 (local address 127.22.63.126:46399)
0806 01:08:20.441264 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:20.441269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:20.441679 (+ 410us) server_negotiation.cc:407] Connection header received
0806 01:08:20.441813 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:20.441817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:20.441880 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:20.441959 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:20.442702 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.443425 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.445326 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.445784 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.445880 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.450321 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.66:36125: BlockingRecv error: recv got EOF from 127.22.63.66:36125 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 06 01:08:20 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:20 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.67@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:20.787719 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:20.777693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.67:37305)
0806 01:08:20.778173 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:20.778193 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:20.778337 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:20.778712 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:20.778715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:20.779182 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:20.779344 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:20.779353 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.780961 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.780966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:20.781929 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:20.781945 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.783024 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.783029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:20.783227 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.784005 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:20.784022 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:20.787487 (+ 3465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":352,"threads_started":1}
W20260806 01:08:20.787998 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:20.777997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:37305 (local address 127.22.63.126:46399)
0806 01:08:20.778198 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:20.778206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:20.778264 (+ 58us) server_negotiation.cc:407] Connection header received
0806 01:08:20.778434 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:20.778440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:20.778558 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:20.778692 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:20.779502 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.780727 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.782160 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:20.782837 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:20.783011 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:20.787688 (+ 4677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.67:37305: BlockingRecv error: recv got EOF from 127.22.63.67:37305 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 01:08:21 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 01:08:21 dist-test-slave-7f01 krb5kdc[2294](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.63.65@KRBTEST.COM for oryx/127.22.63.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 01:08:21.019492 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:21.006617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:46399 (local address 127.22.63.65:60803)
0806 01:08:21.007147 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:21.007164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:21.007283 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:21.007610 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:21.007613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:21.008039 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:21.008224 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:21.008233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.009229 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.009232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:21.010162 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:21.010173 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.010915 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.010919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:21.011054 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:21.015406 (+ 4352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:21.015426 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:21.019175 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.63.126:46399: Server oryx/127.22.63.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
W20260806 01:08:21.019685 3938 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:21.006763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:60803 (local address 127.22.63.126:46399)
0806 01:08:21.007103 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:21.007112 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:21.007270 (+ 158us) server_negotiation.cc:407] Connection header received
0806 01:08:21.007390 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:21.007394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:21.007479 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:21.007568 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:21.008346 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.009088 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.010311 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.010781 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.010892 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:21.019444 (+ 8552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.63.65:60803: BlockingRecv error: recv got EOF from 127.22.63.65:60803 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
I20260806 01:08:21.358903 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3248
I20260806 01:08:21.373199 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:21.360337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.22.63.65:43505)
0806 01:08:21.361059 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:21.361067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:21.361098 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:08:21.361161 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:21.361168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:21.361402 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:21.361591 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:21.363075 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.364295 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.365702 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.366292 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.366405 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:21.369399 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:21.369412 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:21.369434 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:21.369519 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:21.372551 (+ 3032us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:08:21.372884 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42930: Request ticket server kudu/127.22.63.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":404,"threads_started":1}
W20260806 01:08:21.373330 4007 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42930: Request ticket server kudu/127.22.63.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:08:21.373412 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3248: Not authorized: Client connection negotiation failed: client connection to 127.22.63.65:43505: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:08:21.373646 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 3248
I20260806 01:08:21.406675 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3508
W20260806 01:08:21.419801 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3508: Not authorized: Client connection negotiation failed: client connection to 127.22.63.66:46825: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:08:21.419880 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:21.408017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.22.63.66:46825)
0806 01:08:21.408683 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:21.408690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:21.408717 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:08:21.408782 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:21.408788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:21.409058 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:21.409292 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:21.410640 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.411699 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.413011 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.413584 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.413741 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:21.416341 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:21.416350 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:21.416364 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:21.416404 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:21.419031 (+ 2627us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:08:21.419440 (+ 409us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48126: Request ticket server kudu/127.22.63.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":332,"threads_started":1}
W20260806 01:08:21.419970 4008 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48126: Request ticket server kudu/127.22.63.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:08:21.420009 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 3508
I20260806 01:08:21.444099 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3722
I20260806 01:08:21.457738 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:21.445322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.22.63.67:42763)
0806 01:08:21.446070 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:21.446077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:21.446115 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:08:21.446184 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:21.446191 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:21.446437 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:21.446653 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:21.448174 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.449196 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.450439 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:21.450999 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:21.451156 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:21.453897 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:21.453906 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:21.453920 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:21.453968 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:21.456996 (+ 3028us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 01:08:21.457385 (+ 389us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59408: Request ticket server kudu/127.22.63.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":403,"threads_started":1}
W20260806 01:08:21.457768 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3722: Not authorized: Client connection negotiation failed: client connection to 127.22.63.67:42763: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.63.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 01:08:21.457865 4009 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59408: Request ticket server kudu/127.22.63.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 01:08:21.457916 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 3722
I20260806 01:08:21.485838 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 3142
I20260806 01:08:21.487530 3236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:21.609125 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 3142
2026-08-06T01:08:21Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22476 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T01:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:08:21Z Disabled control of system clock
I20260806 01:08:21.707330 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37783
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37783
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 01:08:22.060926 4017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:22.061233 4017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:22.061313 4017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:22.068799 4017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:08:22.068912 4017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:22.068953 4017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:22.069010 4017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:08:22.069056 4017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:08:22.078446 4017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37783
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:22.079809 4017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:22.081606 4017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:22.094022 4024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:22.093990 4029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:22.094066 4025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:22.094743 4017 server_base.cc:1034] running on GCE node
I20260806 01:08:22.095549 4017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:22.097012 4017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:22.098243 4017 hybrid_clock.cc:648] HybridClock initialized: now 1785978502098206 us; error 38 us; skew 500 ppm
I20260806 01:08:22.098594 4017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:22.100963 4017 webserver.cc:460] Webserver started at http://127.22.63.126:44859/ using document root <none> and password file <none>
I20260806 01:08:22.101687 4017 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:22.101822 4017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:22.102164 4017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:22.105159 4017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "99c8353b05b84a9b965aba7b9c30656d"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "2b2c3fc7f30aedbcf11b2adb3a26277e"
server_key_iv: "483a593588eb3ad70afb31b29c54413b"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.105958 4017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "99c8353b05b84a9b965aba7b9c30656d"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "2b2c3fc7f30aedbcf11b2adb3a26277e"
server_key_iv: "483a593588eb3ad70afb31b29c54413b"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.111426 4017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:08:22.115020 4041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:22.116439 4017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:22.116640 4017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "99c8353b05b84a9b965aba7b9c30656d"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "2b2c3fc7f30aedbcf11b2adb3a26277e"
server_key_iv: "483a593588eb3ad70afb31b29c54413b"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.116881 4017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:22.135660 4017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:22.136628 4017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:22.136958 4017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:22.155660 4017 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37783
I20260806 01:08:22.155683 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37783 every 8 connection(s)
I20260806 01:08:22.157213 4017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:08:22.160920 4128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:22.165303 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4017
I20260806 01:08:22.165547 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:08:22.166215 22781 external_mini_cluster.cc:1428] Setting key 010615edd920c796db3100f1100c0d54
I20260806 01:08:22.172237 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d: Bootstrap starting.
I20260806 01:08:22.175277 4128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:22.176409 4128 log.cc:824] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:22.179337 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d: No bootstrap required, opened a new log
I20260806 01:08:22.180259 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:22.168372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.22.63.126:37783)
0806 01:08:22.169427 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:22.169439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:22.169484 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:08:22.171455 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:22.171481 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:22.172030 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:22.172414 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:22.173057 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.174325 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.175775 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.176451 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.176668 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:22.177869 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:22.177876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:22.177890 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:22.178422 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:22.178500 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:22.178520 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:22.178658 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:22.178883 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 01:08:22.179196 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":345,"threads_started":1}
I20260806 01:08:22.187482 4128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } }
I20260806 01:08:22.188030 4128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:22.188140 4128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99c8353b05b84a9b965aba7b9c30656d, State: Initialized, Role: FOLLOWER
I20260806 01:08:22.188704 4128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } }
I20260806 01:08:22.188951 4128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:08:22.189111 4128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:08:22.189301 4128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:22.191536 4128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } }
I20260806 01:08:22.192102 4128 leader_election.cc:302] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99c8353b05b84a9b965aba7b9c30656d; no voters:
I20260806 01:08:22.192792 4128 leader_election.cc:288] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:08:22.192934 4135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:08:22.193848 4135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [term 1 LEADER]: Becoming Leader. State: Replica: 99c8353b05b84a9b965aba7b9c30656d, State: Running, Role: LEADER
I20260806 01:08:22.194518 4135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } }
I20260806 01:08:22.195163 4128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:08:22.197952 4137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99c8353b05b84a9b965aba7b9c30656d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } } }
I20260806 01:08:22.197948 4138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99c8353b05b84a9b965aba7b9c30656d. Latest consensus state: current_term: 1 leader_uuid: "99c8353b05b84a9b965aba7b9c30656d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99c8353b05b84a9b965aba7b9c30656d" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37783 } } }
I20260806 01:08:22.198401 4137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:22.198436 4138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:22.200837 4143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:08:22.207535 4143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:08:22.217712 4143 catalog_manager.cc:1347] Generated new cluster ID: 099aa97b5e6c41bab0022026e44415f4
I20260806 01:08:22.217859 4143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:08:22.236435 4143 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:08:22.238394 4143 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:08:22.253369 4143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99c8353b05b84a9b965aba7b9c30656d: Generated new TSK 0
I20260806 01:08:22.254483 4143 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:08:22.264689 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--builtin_ntp_servers=127.22.63.84:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 01:08:22.628860 4161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:22.629155 4161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:22.629238 4161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:22.636653 4161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:22.636765 4161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:22.636874 4161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:08:22.646411 4161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:22.648339 4161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:22.650278 4161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:22.664784 4169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:22.664750 4170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:22.664683 4172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:22.665887 4161 server_base.cc:1034] running on GCE node
I20260806 01:08:22.666829 4161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:22.667960 4161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:22.669201 4161 hybrid_clock.cc:648] HybridClock initialized: now 1785978502669137 us; error 60 us; skew 500 ppm
I20260806 01:08:22.669553 4161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:22.672152 4161 webserver.cc:460] Webserver started at http://127.22.63.65:42831/ using document root <none> and password file <none>
I20260806 01:08:22.672885 4161 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:22.673031 4161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:22.673380 4161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:22.676416 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "a871bb87fa0b42059619b9f9c14dd194"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "d5e140901c721730cfa2d15cd5ed5a60"
server_key_iv: "18b9377c0475153827c7b63a71d4b0af"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.677245 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "a871bb87fa0b42059619b9f9c14dd194"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "d5e140901c721730cfa2d15cd5ed5a60"
server_key_iv: "18b9377c0475153827c7b63a71d4b0af"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.682612 4161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:08:22.686151 4178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:22.687482 4161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:08:22.687659 4161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "a871bb87fa0b42059619b9f9c14dd194"
format_stamp: "Formatted at 2026-08-06 01:08:22 on dist-test-slave-7f01"
server_key: "d5e140901c721730cfa2d15cd5ed5a60"
server_key_iv: "18b9377c0475153827c7b63a71d4b0af"
server_key_version: "encryptionkey@0"
I20260806 01:08:22.687943 4161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:22.702956 4161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:22.703976 4161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:22.704296 4161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:22.705293 4161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:22.706800 4161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:22.706954 4161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:22.707100 4161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:22.707190 4161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:22.742137 4161 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:39555
I20260806 01:08:22.742197 4345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:39555 every 8 connection(s)
I20260806 01:08:22.743557 4161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:08:22.745985 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4161
I20260806 01:08:22.746208 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:08:22.747027 22781 external_mini_cluster.cc:1428] Setting key ffcb6aba36583d1ae588fb76ffc7704a
I20260806 01:08:22.750362 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--builtin_ntp_servers=127.22.63.84:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:08:22.758981 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:22.746479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:46143 (local address 127.22.63.126:37783)
0806 01:08:22.747091 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:22.747098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:22.748454 (+ 1356us) server_negotiation.cc:407] Connection header received
0806 01:08:22.750855 (+ 2401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:22.750859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:22.750947 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:22.751041 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:22.752594 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.753350 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.755854 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.756351 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.756456 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:22.758159 (+ 1703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:22.758164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:22.758167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:22.758187 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:22.758230 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:22.758242 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:22.758324 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:22.758593 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 01:08:22.758675 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":282,"threads_started":1}
I20260806 01:08:22.760478 4351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:22.747171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37783 (local address 127.22.63.65:46143)
0806 01:08:22.748290 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:22.748349 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:22.750524 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:22.751328 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:22.751342 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:22.751468 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:22.752401 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:22.752432 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.753543 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.753550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:22.755666 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:22.755684 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:22.756594 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:22.756605 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:22.756952 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:22.757753 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:22.757758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:22.757814 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:22.757930 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:22.758346 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:22.758359 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:22.758370 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:08:22.758564 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 01:08:22.759015 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":321,"threads_started":1}
I20260806 01:08:22.762162 4346 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37783
I20260806 01:08:22.762519 4346 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:22.763631 4346 heartbeater.cc:511] Master 127.22.63.126:37783 requested a full tablet report, sending...
I20260806 01:08:22.766064 4077 ts_manager.cc:194] Registered new tserver with Master: a871bb87fa0b42059619b9f9c14dd194 (127.22.63.65:39555)
I20260806 01:08:22.767681 4077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.65:46143
W20260806 01:08:23.106254 4352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:23.106556 4352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:23.106701 4352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:23.114286 4352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:23.114392 4352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:23.114500 4352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:08:23.123847 4352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:23.125129 4352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:23.126891 4352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:23.140012 4357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:23.139974 4361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:23.140043 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:23.141129 4352 server_base.cc:1034] running on GCE node
I20260806 01:08:23.141749 4352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:23.142717 4352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:23.143973 4352 hybrid_clock.cc:648] HybridClock initialized: now 1785978503143908 us; error 54 us; skew 500 ppm
I20260806 01:08:23.144292 4352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:23.146698 4352 webserver.cc:460] Webserver started at http://127.22.63.66:41823/ using document root <none> and password file <none>
I20260806 01:08:23.147435 4352 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:23.147569 4352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:23.147954 4352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:23.150785 4352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "e3686efa2baf4261ae9ed8f6c3c7c551"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "a2ed6537fbe9f6ad27ec787d196789cf"
server_key_iv: "72a9d59665d0971ead0742eb30e12367"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.151590 4352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "e3686efa2baf4261ae9ed8f6c3c7c551"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "a2ed6537fbe9f6ad27ec787d196789cf"
server_key_iv: "72a9d59665d0971ead0742eb30e12367"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.156821 4352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 01:08:23.160444 4373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.161859 4352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:08:23.162077 4352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "e3686efa2baf4261ae9ed8f6c3c7c551"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "a2ed6537fbe9f6ad27ec787d196789cf"
server_key_iv: "72a9d59665d0971ead0742eb30e12367"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.162317 4352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:23.184141 4352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:23.185032 4352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:23.185355 4352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:23.186444 4352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:23.187945 4352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:23.188037 4352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.188118 4352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:23.188176 4352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.223003 4352 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39977
I20260806 01:08:23.223203 4547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39977 every 8 connection(s)
I20260806 01:08:23.224340 4352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:08:23.234380 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4352
I20260806 01:08:23.234632 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:08:23.235086 22781 external_mini_cluster.cc:1428] Setting key 88c74f1dd1c3dc870dc65257334da3e5
I20260806 01:08:23.236972 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.227004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:45259 (local address 127.22.63.126:37783)
0806 01:08:23.227195 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.227200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.228593 (+ 1393us) server_negotiation.cc:407] Connection header received
0806 01:08:23.230353 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.230361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.230458 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.230573 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.231622 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.232431 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.233989 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.234399 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.234540 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.236091 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:23.236116 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:23.236122 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:23.236144 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:23.236184 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:23.236199 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:23.236294 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:23.236531 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 01:08:23.236623 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:08:23.237977 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.227522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37783 (local address 127.22.63.66:45259)
0806 01:08:23.228435 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:23.228471 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:23.230084 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:23.230797 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:23.230805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:23.230884 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:23.231454 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:23.231471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.232603 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.232608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:23.233834 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:23.233847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.234695 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.234702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:23.234956 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.235704 (+ 748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:23.235707 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:23.235791 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:23.235888 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:23.236312 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:23.236321 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:23.236328 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:08:23.236491 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 01:08:23.236796 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":314,"threads_started":1}
I20260806 01:08:23.238121 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--builtin_ntp_servers=127.22.63.84:36561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 01:08:23.239398 4548 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37783
I20260806 01:08:23.239709 4548 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:23.240549 4548 heartbeater.cc:511] Master 127.22.63.126:37783 requested a full tablet report, sending...
I20260806 01:08:23.241998 4077 ts_manager.cc:194] Registered new tserver with Master: e3686efa2baf4261ae9ed8f6c3c7c551 (127.22.63.66:39977)
I20260806 01:08:23.242787 4077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.66:45259
W20260806 01:08:23.587677 4551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:23.588011 4551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:23.588095 4551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:23.595562 4551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:23.595679 4551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:23.595845 4551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:08:23.605124 4551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37783
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:23.606496 4551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:23.608546 4551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:23.622361 4558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:23.622423 4557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:23.622583 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:23.623322 4551 server_base.cc:1034] running on GCE node
I20260806 01:08:23.624043 4551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:23.625041 4551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:23.626250 4551 hybrid_clock.cc:648] HybridClock initialized: now 1785978503626200 us; error 37 us; skew 500 ppm
I20260806 01:08:23.626601 4551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:23.629226 4551 webserver.cc:460] Webserver started at http://127.22.63.67:46179/ using document root <none> and password file <none>
I20260806 01:08:23.630010 4551 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:23.630106 4551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:23.630496 4551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:23.633591 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "c5ba564b8df64f7fac2784cae659496f"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "ca7f415700cbf91435ae171108281b0a"
server_key_iv: "b1ef0dcde303db9abfae9bd5a781ad91"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.634459 4551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "c5ba564b8df64f7fac2784cae659496f"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "ca7f415700cbf91435ae171108281b0a"
server_key_iv: "b1ef0dcde303db9abfae9bd5a781ad91"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.640053 4551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260806 01:08:23.643579 4578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.645002 4551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:23.645203 4551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "c5ba564b8df64f7fac2784cae659496f"
format_stamp: "Formatted at 2026-08-06 01:08:23 on dist-test-slave-7f01"
server_key: "ca7f415700cbf91435ae171108281b0a"
server_key_iv: "b1ef0dcde303db9abfae9bd5a781ad91"
server_key_version: "encryptionkey@0"
I20260806 01:08:23.645442 4551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:23.675664 4551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:23.676802 4551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:23.677071 4551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:23.678054 4551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:23.679629 4551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:23.679770 4551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.679898 4551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:23.679993 4551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:23.714602 4551 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:36793
I20260806 01:08:23.714704 4737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:36793 every 8 connection(s)
I20260806 01:08:23.716069 4551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:08:23.719408 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4551
I20260806 01:08:23.719630 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:08:23.720216 22781 external_mini_cluster.cc:1428] Setting key e0556b7d2ae1d33e1f843d3b22023120
I20260806 01:08:23.729362 4350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.718765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:50999 (local address 127.22.63.126:37783)
0806 01:08:23.718975 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.718979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.720553 (+ 1574us) server_negotiation.cc:407] Connection header received
0806 01:08:23.722626 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.722634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.722722 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.722840 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.724087 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.724830 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.726382 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.726878 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.726991 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.728583 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:23.728588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:23.728591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:23.728610 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:23.728641 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:23.728652 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:23.728726 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:23.729018 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 01:08:23.729103 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:08:23.730368 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.719241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37783 (local address 127.22.63.67:50999)
0806 01:08:23.720369 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:23.720415 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:23.722390 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:23.723044 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:23.723052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:23.723143 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:23.723916 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:23.723936 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.724992 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.724995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:23.726220 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:23.726232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.727115 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.727120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:23.727325 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.728266 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:23.728269 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:23.728314 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:23.728396 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:23.728739 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:23.728747 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:23.728753 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:08:23.728939 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 01:08:23.729245 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":347,"threads_started":1}
I20260806 01:08:23.731870 4738 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37783
I20260806 01:08:23.732187 4738 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:23.732990 4738 heartbeater.cc:511] Master 127.22.63.126:37783 requested a full tablet report, sending...
I20260806 01:08:23.734551 4077 ts_manager.cc:194] Registered new tserver with Master: c5ba564b8df64f7fac2784cae659496f (127.22.63.67:36793)
I20260806 01:08:23.735572 4077 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.67:50999
I20260806 01:08:23.736244 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:08:23.742776 4350 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:23.740953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.22.63.126:37783)
0806 01:08:23.741126 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.741130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.741733 (+ 603us) server_negotiation.cc:407] Connection header received
0806 01:08:23.741905 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.741909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.741987 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.742083 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.742563 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38364: BlockingRecv error: recv got EOF from 127.0.0.1:38364 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:08:23.743299 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4161
I20260806 01:08:23.752220 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.744462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.22.63.65:39555)
0806 01:08:23.745162 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.745169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.745194 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:08:23.745247 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.745253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.745447 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.745577 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.746108 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.747187 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.748909 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.749437 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.749539 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.750687 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:23.750695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:23.750707 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:23.751268 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:23.751316 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:23.751342 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:23.751436 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:23.751646 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 01:08:23.751885 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":394,"threads_started":1}
I20260806 01:08:23.753458 4338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:23.836304 4346 heartbeater.cc:503] Master 127.22.63.126:37783 was elected leader, sending a full tablet report...
I20260806 01:08:23.837481 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4161
W20260806 01:08:23.866114 4045 connection.cc:535] server connection from 127.22.63.65:46143 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 01:08:23.866248 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4352
I20260806 01:08:23.874616 4758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.867241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46566 (local address 127.22.63.66:39977)
0806 01:08:23.867865 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.867872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.867906 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:08:23.867980 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.867987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.868210 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.868353 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.868834 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.869868 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.871102 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.871690 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.871885 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.872991 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:23.872999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:23.873013 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:23.873814 (+ 801us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:23.873882 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:23.873913 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:23.874006 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:23.874187 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 01:08:23.874302 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":315,"threads_started":1}
I20260806 01:08:23.875963 4542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:23.967608 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4352
I20260806 01:08:23.991173 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4551
I20260806 01:08:24.000178 4760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:23.992477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.22.63.67:36793)
0806 01:08:23.993187 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:23.993202 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:23.993233 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:08:23.993295 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:23.993302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:23.993539 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:23.993732 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:23.994237 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.995300 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.996788 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:23.997361 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:23.997513 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:23.998664 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:23.998673 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:23.998687 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:23.999294 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:23.999343 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:23.999363 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:23.999522 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:23.999635 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:08:23.999804 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":379,"threads_started":1}
I20260806 01:08:24.001490 4731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:24.084410 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4551
I20260806 01:08:24.108412 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4017
I20260806 01:08:24.109691 4119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:24.214284 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4017
2026-08-06T01:08:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2599 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T01:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:08:24Z Disabled control of system clock
I20260806 01:08:24.309844 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:36499
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:36499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:08:24.655002 4770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:24.655311 4770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:24.655398 4770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:24.662616 4770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:08:24.662727 4770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:24.662765 4770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:24.662822 4770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:08:24.662865 4770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:08:24.672140 4770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:36499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:36499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:24.673453 4770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:24.675302 4770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:24.687916 4777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:24.687880 4782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:24.687841 4775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:24.688671 4770 server_base.cc:1034] running on GCE node
I20260806 01:08:24.689510 4770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:24.690933 4770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:24.692174 4770 hybrid_clock.cc:648] HybridClock initialized: now 1785978504692145 us; error 54 us; skew 500 ppm
I20260806 01:08:24.692556 4770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:24.694969 4770 webserver.cc:460] Webserver started at http://127.22.63.126:45115/ using document root <none> and password file <none>
I20260806 01:08:24.695706 4770 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:24.695892 4770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:24.696233 4770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:24.701210 4770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "9dd6be66b10341bd8431796fd6b04b81"
format_stamp: "Formatted at 2026-08-06 01:08:24 on dist-test-slave-7f01"
server_key: "1a24ae87d2696888a062ca1f68f1f5c5"
server_key_iv: "a18f600d9025a8b67b2b825b371ce027"
server_key_version: "encryptionkey@0"
I20260806 01:08:24.702046 4770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "9dd6be66b10341bd8431796fd6b04b81"
format_stamp: "Formatted at 2026-08-06 01:08:24 on dist-test-slave-7f01"
server_key: "1a24ae87d2696888a062ca1f68f1f5c5"
server_key_iv: "a18f600d9025a8b67b2b825b371ce027"
server_key_version: "encryptionkey@0"
I20260806 01:08:24.707355 4770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:08:24.710912 4794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:24.712352 4770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:24.712560 4770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "9dd6be66b10341bd8431796fd6b04b81"
format_stamp: "Formatted at 2026-08-06 01:08:24 on dist-test-slave-7f01"
server_key: "1a24ae87d2696888a062ca1f68f1f5c5"
server_key_iv: "a18f600d9025a8b67b2b825b371ce027"
server_key_version: "encryptionkey@0"
I20260806 01:08:24.712795 4770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:24.718966 4770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:24.719884 4770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:24.720201 4770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:24.739305 4770 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:36499
I20260806 01:08:24.739347 4873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:36499 every 8 connection(s)
I20260806 01:08:24.740772 4770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:08:24.744745 4875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:24.750289 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4770
I20260806 01:08:24.750499 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:08:24.751343 22781 external_mini_cluster.cc:1428] Setting key 300e84adf84342a28a48e03542dbdfef
I20260806 01:08:24.756548 4875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81: Bootstrap starting.
I20260806 01:08:24.760221 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:24.753686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.22.63.126:36499)
0806 01:08:24.754806 (+ 1120us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:24.754817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:24.754865 (+ 48us) server_negotiation.cc:407] Connection header received
0806 01:08:24.756663 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:24.756693 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:24.757096 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:24.757404 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:24.758579 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:24.758585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:24.758598 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:24.758611 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:24.758671 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:24.758686 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:24.758858 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:24.759003 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 01:08:24.759267 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":426,"threads_started":1}
I20260806 01:08:24.760268 4875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:24.761407 4875 log.cc:824] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:24.764358 4875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81: No bootstrap required, opened a new log
I20260806 01:08:24.771399 4875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } }
I20260806 01:08:24.771795 4875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:24.771900 4875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dd6be66b10341bd8431796fd6b04b81, State: Initialized, Role: FOLLOWER
I20260806 01:08:24.772523 4875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } }
I20260806 01:08:24.772755 4875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:08:24.772908 4875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:08:24.773089 4875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:24.775377 4875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } }
I20260806 01:08:24.775993 4875 leader_election.cc:302] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dd6be66b10341bd8431796fd6b04b81; no voters:
I20260806 01:08:24.776635 4875 leader_election.cc:288] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:08:24.776824 4881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:08:24.778075 4881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [term 1 LEADER]: Becoming Leader. State: Replica: 9dd6be66b10341bd8431796fd6b04b81, State: Running, Role: LEADER
I20260806 01:08:24.778590 4881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } }
I20260806 01:08:24.778805 4875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:08:24.781333 4884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dd6be66b10341bd8431796fd6b04b81. Latest consensus state: current_term: 1 leader_uuid: "9dd6be66b10341bd8431796fd6b04b81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } } }
I20260806 01:08:24.781710 4884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:24.782142 4882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dd6be66b10341bd8431796fd6b04b81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dd6be66b10341bd8431796fd6b04b81" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36499 } } }
I20260806 01:08:24.782608 4882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81 [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:24.783357 4889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:08:24.789830 4889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:08:24.799669 4889 catalog_manager.cc:1347] Generated new cluster ID: 7420f2275a4948728baed36b0f0007e9
I20260806 01:08:24.799846 4889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:08:24.810324 4889 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:08:24.810823 4889 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:08:24.829367 4889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9dd6be66b10341bd8431796fd6b04b81: Generated new TSK 0
I20260806 01:08:24.830264 4889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:08:24.844133 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--builtin_ntp_servers=127.22.63.84:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 01:08:25.201324 4908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:25.201606 4908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:25.201797 4908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:25.209115 4908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:25.209213 4908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:25.209410 4908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:08:25.218817 4908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:25.220232 4908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:25.222090 4908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:25.236070 4915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:25.235965 4919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:25.236039 4916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:25.237193 4908 server_base.cc:1034] running on GCE node
I20260806 01:08:25.237757 4908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:25.238747 4908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:25.239962 4908 hybrid_clock.cc:648] HybridClock initialized: now 1785978505239903 us; error 52 us; skew 500 ppm
I20260806 01:08:25.240309 4908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:25.242720 4908 webserver.cc:460] Webserver started at http://127.22.63.65:45673/ using document root <none> and password file <none>
I20260806 01:08:25.243458 4908 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:25.243582 4908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:25.244014 4908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:25.247071 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "c66945138671465bbb22feda0de2b6d2"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "2c55b92b8ba30e76e4666a3266d9e416"
server_key_iv: "40b45946785ecbf28506bbed1bb8cf44"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.247989 4908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "c66945138671465bbb22feda0de2b6d2"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "2c55b92b8ba30e76e4666a3266d9e416"
server_key_iv: "40b45946785ecbf28506bbed1bb8cf44"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.253293 4908 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:08:25.256958 4929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.258415 4908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:08:25.258597 4908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "c66945138671465bbb22feda0de2b6d2"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "2c55b92b8ba30e76e4666a3266d9e416"
server_key_iv: "40b45946785ecbf28506bbed1bb8cf44"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.258829 4908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:25.265084 4908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:25.265991 4908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:25.266281 4908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:25.267227 4908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:25.268777 4908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:25.268901 4908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.269037 4908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:25.269138 4908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.304203 4908 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:37341
I20260806 01:08:25.304284 5116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:37341 every 8 connection(s)
I20260806 01:08:25.305617 4908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:08:25.306566 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 4908
I20260806 01:08:25.306790 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:08:25.307528 22781 external_mini_cluster.cc:1428] Setting key 067f9301a189245cce4c40184cf3ce3c
I20260806 01:08:25.310791 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--builtin_ntp_servers=127.22.63.84:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:08:25.314671 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:25.308489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:51937 (local address 127.22.63.126:36499)
0806 01:08:25.308957 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:25.308964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:25.310413 (+ 1449us) server_negotiation.cc:407] Connection header received
0806 01:08:25.312264 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:25.312273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:25.312349 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:25.312472 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:25.313820 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:25.313828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:25.313836 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:25.313860 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:25.313927 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:25.313945 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:25.314074 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:25.314241 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 01:08:25.314311 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":209,"threads_started":1}
I20260806 01:08:25.315500 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:25.309139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36499 (local address 127.22.63.65:51937)
0806 01:08:25.310244 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:25.310290 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:25.311989 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:25.312679 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:25.312687 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:25.312781 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:25.313512 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:25.313514 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:25.313554 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:25.313640 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:25.314047 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:25.314055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:25.314060 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 01:08:25.314274 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 01:08:25.314486 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":369,"threads_started":1}
I20260806 01:08:25.317129 5117 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36499
I20260806 01:08:25.317443 5117 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:25.318044 5117 heartbeater.cc:511] Master 127.22.63.126:36499 requested a full tablet report, sending...
I20260806 01:08:25.320247 4821 ts_manager.cc:194] Registered new tserver with Master: c66945138671465bbb22feda0de2b6d2 (127.22.63.65:37341)
W20260806 01:08:25.667438 5126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:25.667809 5126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:25.667976 5126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:25.675343 5126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:25.675457 5126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:25.675640 5126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:08:25.685902 5126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:25.687331 5126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:25.689255 5126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:25.702778 5134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:25.703017 5139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:25.702865 5133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:25.703941 5126 server_base.cc:1034] running on GCE node
I20260806 01:08:25.704576 5126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:25.705556 5126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:25.706808 5126 hybrid_clock.cc:648] HybridClock initialized: now 1785978505706742 us; error 70 us; skew 500 ppm
I20260806 01:08:25.707165 5126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:25.709491 5126 webserver.cc:460] Webserver started at http://127.22.63.66:43487/ using document root <none> and password file <none>
I20260806 01:08:25.710219 5126 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:25.710359 5126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:25.710698 5126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:25.713727 5126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "f08371b5b5c44e86b8ae75c8217c41af"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "9e0c5ffbc0a27e0fe51d2a1dd6a57672"
server_key_iv: "03cc6ac240c1bb626dfa4fe888efea42"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.714581 5126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "f08371b5b5c44e86b8ae75c8217c41af"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "9e0c5ffbc0a27e0fe51d2a1dd6a57672"
server_key_iv: "03cc6ac240c1bb626dfa4fe888efea42"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.720315 5126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260806 01:08:25.724050 5151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.725580 5126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:08:25.725793 5126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "f08371b5b5c44e86b8ae75c8217c41af"
format_stamp: "Formatted at 2026-08-06 01:08:25 on dist-test-slave-7f01"
server_key: "9e0c5ffbc0a27e0fe51d2a1dd6a57672"
server_key_iv: "03cc6ac240c1bb626dfa4fe888efea42"
server_key_version: "encryptionkey@0"
I20260806 01:08:25.726058 5126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:25.732170 5126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:25.733006 5126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:25.733299 5126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:25.734237 5126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:25.735697 5126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:25.735858 5126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.736006 5126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:25.736109 5126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:25.771903 5126 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39699
I20260806 01:08:25.771950 5326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39699 every 8 connection(s)
I20260806 01:08:25.773332 5126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:08:25.782102 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:25.776191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59915 (local address 127.22.63.126:36499)
0806 01:08:25.776377 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:25.776382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:25.777948 (+ 1566us) server_negotiation.cc:407] Connection header received
0806 01:08:25.779675 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:25.779680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:25.779840 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:25.779954 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:25.781461 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:25.781466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:25.781469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:25.781487 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:25.781522 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:25.781534 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:25.781646 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:25.781834 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 01:08:25.781889 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:08:25.783138 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:25.776743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36499 (local address 127.22.63.66:59915)
0806 01:08:25.777774 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:25.777819 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:25.779403 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:25.780296 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:25.780304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:25.780394 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:25.781136 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:25.781138 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:25.781178 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:25.781264 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:25.781627 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:25.781635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:25.781641 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 01:08:25.781861 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 01:08:25.782067 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":286,"threads_started":1}
I20260806 01:08:25.783381 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 5126
I20260806 01:08:25.783560 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:08:25.784140 22781 external_mini_cluster.cc:1428] Setting key b42675d1ea885425cf370037fc8f5c58
I20260806 01:08:25.784855 5327 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36499
I20260806 01:08:25.785207 5327 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:25.785946 5327 heartbeater.cc:511] Master 127.22.63.126:36499 requested a full tablet report, sending...
I20260806 01:08:25.787348 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--builtin_ntp_servers=127.22.63.84:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 01:08:25.787379 4821 ts_manager.cc:194] Registered new tserver with Master: f08371b5b5c44e86b8ae75c8217c41af (127.22.63.66:39699)
W20260806 01:08:26.137845 5331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:26.138139 5331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:26.138229 5331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:26.145381 5331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:26.145478 5331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:26.145588 5331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:08:26.154644 5331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:26.156055 5331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:26.157887 5331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:26.171620 5346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:26.171701 5339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:26.171653 5340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:26.177544 5331 server_base.cc:1034] running on GCE node
I20260806 01:08:26.178300 5331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:26.179466 5331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:26.180665 5331 hybrid_clock.cc:648] HybridClock initialized: now 1785978506180595 us; error 73 us; skew 500 ppm
I20260806 01:08:26.180958 5331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:08:26.188678 5331 webserver.cc:460] Webserver started at http://127.22.63.67:46245/ using document root <none> and password file <none>
I20260806 01:08:26.189414 5331 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:26.189504 5331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:26.189782 5331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:26.192798 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "c2e0c2a4dbce47678c5dc912c6c019c3"
format_stamp: "Formatted at 2026-08-06 01:08:26 on dist-test-slave-7f01"
server_key: "508060fdd7011d140577b3733821a2bb"
server_key_iv: "abb276270b28df9126a8321f792e5f01"
server_key_version: "encryptionkey@0"
I20260806 01:08:26.193567 5331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "c2e0c2a4dbce47678c5dc912c6c019c3"
format_stamp: "Formatted at 2026-08-06 01:08:26 on dist-test-slave-7f01"
server_key: "508060fdd7011d140577b3733821a2bb"
server_key_iv: "abb276270b28df9126a8321f792e5f01"
server_key_version: "encryptionkey@0"
I20260806 01:08:26.198787 5331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:08:26.202282 5357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:26.203719 5331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:08:26.203936 5331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "c2e0c2a4dbce47678c5dc912c6c019c3"
format_stamp: "Formatted at 2026-08-06 01:08:26 on dist-test-slave-7f01"
server_key: "508060fdd7011d140577b3733821a2bb"
server_key_iv: "abb276270b28df9126a8321f792e5f01"
server_key_version: "encryptionkey@0"
I20260806 01:08:26.204129 5331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:26.209759 5331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:26.210520 5331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:26.210747 5331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:26.211611 5331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:26.213048 5331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:26.213138 5331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:26.213222 5331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:26.213279 5331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:26.248646 5331 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:35765
I20260806 01:08:26.248682 5543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:35765 every 8 connection(s)
I20260806 01:08:26.250020 5331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:08:26.258280 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 5331
I20260806 01:08:26.258436 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:26.252808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:55729 (local address 127.22.63.126:36499)
0806 01:08:26.253055 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:26.253061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:26.254431 (+ 1370us) server_negotiation.cc:407] Connection header received
0806 01:08:26.256195 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:26.256200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:26.256268 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:26.256358 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:26.257698 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:26.257702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:26.257705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:26.257721 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:26.257754 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:26.257765 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:26.257874 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:26.258137 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 01:08:26.258190 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:08:26.258481 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:08:26.259068 22781 external_mini_cluster.cc:1428] Setting key 7aaa4ad7fd2b373e2f5d9959120b8891
I20260806 01:08:26.259783 5546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:26.253340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36499 (local address 127.22.63.67:55729)
0806 01:08:26.254267 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:26.254312 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:26.255927 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:26.256608 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:26.256616 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:26.256708 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:26.257397 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:08:26.257400 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:26.257439 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:08:26.257527 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:26.257873 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:26.257887 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:26.257898 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:08:26.258229 (+ 331us) client_negotiation.cc:240] Negotiation successful
0806 01:08:26.258512 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":287,"threads_started":1}
I20260806 01:08:26.261293 5544 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36499
I20260806 01:08:26.261608 5544 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:26.262308 5544 heartbeater.cc:511] Master 127.22.63.126:36499 requested a full tablet report, sending...
I20260806 01:08:26.263926 4821 ts_manager.cc:194] Registered new tserver with Master: c2e0c2a4dbce47678c5dc912c6c019c3 (127.22.63.67:35765)
I20260806 01:08:26.274884 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 01:08:26.281700 5121 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:08:26.279972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.22.63.126:36499)
0806 01:08:26.280197 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:26.280206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:26.280772 (+ 566us) server_negotiation.cc:407] Connection header received
0806 01:08:26.280914 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:26.280919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:26.280976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:26.281062 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:26.281424 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55354: BlockingRecv error: recv got EOF from 127.0.0.1:55354 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:08:26.282383 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4908
I20260806 01:08:26.286078 5555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:26.283307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.22.63.65:37341)
0806 01:08:26.283880 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:26.283888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:26.283916 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:08:26.283977 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:26.283983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:26.284195 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:26.284322 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:26.285343 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:26.285349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:26.285360 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:26.285372 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:26.285414 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:26.285431 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:26.285550 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:26.285738 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 01:08:26.285830 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":281,"threads_started":1}
I20260806 01:08:26.287357 5111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:26.369681 5117 heartbeater.cc:503] Master 127.22.63.126:36499 was elected leader, sending a full tablet report...
I20260806 01:08:26.371608 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4908
I20260806 01:08:26.399675 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 5126
I20260806 01:08:26.403425 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:26.400734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.22.63.66:39699)
0806 01:08:26.401258 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:26.401266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:26.401293 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:08:26.401360 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:26.401367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:26.401605 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:26.401736 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:26.402683 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:26.402689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:26.402700 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:26.402718 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:26.402758 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:26.402772 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:26.402879 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:26.402993 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 01:08:26.403121 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":234,"threads_started":1}
I20260806 01:08:26.404847 5317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:26.487815 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 5126
I20260806 01:08:26.523581 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 5331
I20260806 01:08:26.529078 5560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:26.524740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.22.63.67:35765)
0806 01:08:26.525682 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:26.525695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:26.525745 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:08:26.525851 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:26.525864 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:26.526211 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:26.526414 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:26.528002 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:26.528015 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:26.528037 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:08:26.528067 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:26.528137 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:08:26.528162 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:26.528377 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:26.528466 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 01:08:26.528590 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":470,"threads_started":1}
I20260806 01:08:26.530471 5535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:26.621410 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 5331
I20260806 01:08:26.646404 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 4770
I20260806 01:08:26.647794 4868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:26.752985 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 4770
2026-08-06T01:08:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2538 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:08:26 dist-test-slave-7f01 krb5kdc[5575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:08:26 dist-test-slave-7f01 krb5kdc[5575](info): set up 2 sockets
Aug 06 01:08:26 dist-test-slave-7f01 krb5kdc[5575](info): commencing operation
krb5kdc: starting...
W20260806 01:08:28.108546 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:08:28 dist-test-slave-7f01 krb5kdc[5575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:08:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:08:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:08:28.419483 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:36729
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:28.765237 5601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:28.765532 5601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:28.765704 5601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:28.773005 5601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:08:28.773109 5601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:28.773226 5601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:28.773305 5601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:08:28.773373 5601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:08:28.783399 5601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:36729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:28.784878 5601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:28.786331 5601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:28.798794 5609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:28.798856 5610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:28.798933 5614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:28.800182 5601 server_base.cc:1034] running on GCE node
I20260806 01:08:28.800931 5601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:28.802454 5601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:28.803691 5601 hybrid_clock.cc:648] HybridClock initialized: now 1785978508803646 us; error 34 us; skew 500 ppm
I20260806 01:08:28.804162 5601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:28 dist-test-slave-7f01 krb5kdc[5575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:28.807452 5601 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:08:28.809154 5601 webserver.cc:460] Webserver started at http://127.22.63.126:40379/ using document root <none> and password file <none>
I20260806 01:08:28.809857 5601 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:28.810010 5601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:28.810356 5601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:28.813490 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "2785cb39b51d4d2d95d7a964f46591eb"
format_stamp: "Formatted at 2026-08-06 01:08:28 on dist-test-slave-7f01"
server_key: "130aa6b898db2211cf5e390ecc3ad840"
server_key_iv: "db513e14f207f349ed88e394a67408ac"
server_key_version: "encryptionkey@0"
I20260806 01:08:28.814291 5601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "2785cb39b51d4d2d95d7a964f46591eb"
format_stamp: "Formatted at 2026-08-06 01:08:28 on dist-test-slave-7f01"
server_key: "130aa6b898db2211cf5e390ecc3ad840"
server_key_iv: "db513e14f207f349ed88e394a67408ac"
server_key_version: "encryptionkey@0"
I20260806 01:08:28.819602 5601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:08:28.823102 5629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:28.824491 5601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:28.824697 5601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "2785cb39b51d4d2d95d7a964f46591eb"
format_stamp: "Formatted at 2026-08-06 01:08:28 on dist-test-slave-7f01"
server_key: "130aa6b898db2211cf5e390ecc3ad840"
server_key_iv: "db513e14f207f349ed88e394a67408ac"
server_key_version: "encryptionkey@0"
I20260806 01:08:28.824908 5601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:28.852623 5601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:28.857311 5601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:28.857645 5601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:28.877130 5601 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:36729
I20260806 01:08:28.877162 5705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:36729 every 8 connection(s)
I20260806 01:08:28.878618 5601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:08:28.880666 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 5601
I20260806 01:08:28.880889 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:08:28.881592 22781 external_mini_cluster.cc:1428] Setting key 39208c92b2f1083be5741324e610f26a
I20260806 01:08:28.882853 5706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:08:28 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:28.897958 5706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: Bootstrap starting.
I20260806 01:08:28.901453 5706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:28.902767 5706 log.cc:824] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:28.905428 5715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:28.883598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.22.63.126:36729)
0806 01:08:28.884417 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:28.884428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:28.884470 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:08:28.886141 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:28.886162 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:28.886556 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:28.886973 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:28.888935 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:28.890302 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:28.892159 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:28.892827 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:28.892966 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:28.897951 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:28.897959 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:28.897974 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:28.898016 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:28.901133 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:28.901999 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:28.902006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:28.902012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:28.902181 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:28.902615 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:28.902618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:28.902620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:28.903212 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:28.903589 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:28.904031 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 01:08:28.904451 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260806 01:08:28.906021 5706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: No bootstrap required, opened a new log
I20260806 01:08:28.914348 5706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } }
I20260806 01:08:28.914731 5706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:28.914834 5706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2785cb39b51d4d2d95d7a964f46591eb, State: Initialized, Role: FOLLOWER
I20260806 01:08:28.915392 5706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } }
I20260806 01:08:28.915629 5706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:08:28.915781 5706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:08:28.915980 5706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:28.918401 5706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } }
I20260806 01:08:28.919003 5706 leader_election.cc:302] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2785cb39b51d4d2d95d7a964f46591eb; no voters:
I20260806 01:08:28.919639 5706 leader_election.cc:288] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:08:28.919816 5719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:08:28.920521 5719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [term 1 LEADER]: Becoming Leader. State: Replica: 2785cb39b51d4d2d95d7a964f46591eb, State: Running, Role: LEADER
I20260806 01:08:28.921054 5719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } }
I20260806 01:08:28.921792 5706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:08:28.924327 5720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2785cb39b51d4d2d95d7a964f46591eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } } }
I20260806 01:08:28.924703 5720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:28.925896 5721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2785cb39b51d4d2d95d7a964f46591eb. Latest consensus state: current_term: 1 leader_uuid: "2785cb39b51d4d2d95d7a964f46591eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2785cb39b51d4d2d95d7a964f46591eb" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 36729 } } }
I20260806 01:08:28.926259 5721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb [sys.catalog]: This master's current role is: LEADER
I20260806 01:08:28.927048 5729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:08:28.932171 5729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:08:28.941501 5729 catalog_manager.cc:1347] Generated new cluster ID: 01c9a815a57949c29a196a6bb0bc7da8
I20260806 01:08:28.941644 5729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:08:28.984162 5729 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:08:28.985626 5729 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:08:28.993894 5729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: Generated new TSK 0
I20260806 01:08:28.994673 5729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:08:29.079512 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:29.451012 5751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:29.451283 5751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:29.451462 5751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:29.458830 5751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:29.458943 5751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:29.459172 5751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:08:29.469027 5751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:29.470336 5751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:29.471707 5751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:29.485100 5761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:29.485124 5763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:29.485414 5767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:29.486199 5751 server_base.cc:1034] running on GCE node
I20260806 01:08:29.486840 5751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:29.487782 5751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:29.489022 5751 hybrid_clock.cc:648] HybridClock initialized: now 1785978509488975 us; error 40 us; skew 500 ppm
I20260806 01:08:29.489336 5751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:29 dist-test-slave-7f01 krb5kdc[5575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:29.492553 5751 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:08:29.494100 5751 webserver.cc:460] Webserver started at http://127.22.63.65:39507/ using document root <none> and password file <none>
I20260806 01:08:29.494732 5751 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:29.494830 5751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:29.495090 5751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:29.497956 5751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "5025f3788b924c4bba45b576710d94e9"
format_stamp: "Formatted at 2026-08-06 01:08:29 on dist-test-slave-7f01"
server_key: "98c47d2e206b0a1fe8f7f45df644d5f6"
server_key_iv: "b36e712fd0d1f17270840b7c8212c00f"
server_key_version: "encryptionkey@0"
I20260806 01:08:29.498661 5751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "5025f3788b924c4bba45b576710d94e9"
format_stamp: "Formatted at 2026-08-06 01:08:29 on dist-test-slave-7f01"
server_key: "98c47d2e206b0a1fe8f7f45df644d5f6"
server_key_iv: "b36e712fd0d1f17270840b7c8212c00f"
server_key_version: "encryptionkey@0"
I20260806 01:08:29.503921 5751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:08:29.507236 5780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:29.508533 5751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:29.508702 5751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "5025f3788b924c4bba45b576710d94e9"
format_stamp: "Formatted at 2026-08-06 01:08:29 on dist-test-slave-7f01"
server_key: "98c47d2e206b0a1fe8f7f45df644d5f6"
server_key_iv: "b36e712fd0d1f17270840b7c8212c00f"
server_key_version: "encryptionkey@0"
I20260806 01:08:29.508893 5751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:29.521100 5751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:29.525540 5751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:29.525849 5751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:29.526810 5751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:29.528339 5751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:29.528414 5751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:29.528553 5751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:29.528618 5751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:29.564059 5751 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44907
I20260806 01:08:29.564116 5964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44907 every 8 connection(s)
I20260806 01:08:29.565424 5751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:08:29.574838 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 5751
I20260806 01:08:29.575068 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:08:29.575887 22781 external_mini_cluster.cc:1428] Setting key b2ee57040a412035c2ddde77dc6effdc
Aug 06 01:08:29 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978509, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:29.587893 5967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:29.568176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:52983 (local address 127.22.63.126:36729)
0806 01:08:29.568701 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:29.568707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:29.569675 (+ 968us) server_negotiation.cc:407] Connection header received
0806 01:08:29.571405 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:29.571410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:29.571480 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:29.571577 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:29.573166 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:29.573909 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:29.575593 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:29.576262 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:29.576407 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:29.580511 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:29.580520 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:29.580528 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:29.580574 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:29.584163 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:29.584940 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:29.584948 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:29.584953 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:29.585111 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:29.585610 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:29.585618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:29.585624 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:29.586016 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:29.586182 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:29.587105 (+ 923us) server_negotiation.cc:299] Negotiation successful
0806 01:08:29.587377 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":233,"threads_started":1}
I20260806 01:08:29.588591 5968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:29.568723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36729 (local address 127.22.63.65:52983)
0806 01:08:29.569501 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:29.569550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:29.571157 (+ 1607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:29.571887 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:29.571896 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:29.572413 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:29.572999 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:29.573017 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:29.574058 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:29.574062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:29.575428 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:29.575440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:29.576482 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:29.576487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:29.576790 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:29.577545 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:29.577570 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:29.580278 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:29.584326 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:29.584334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:29.584347 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:29.584773 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:29.585281 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:29.585284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:29.585287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:29.585488 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:29.586144 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:29.586150 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:29.586691 (+ 541us) client_negotiation.cc:769] Sending connection context
0806 01:08:29.586947 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 01:08:29.587369 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":290,"threads_started":1}
I20260806 01:08:29.590430 5965 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36729
I20260806 01:08:29.590788 5965 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:29.591572 5965 heartbeater.cc:511] Master 127.22.63.126:36729 requested a full tablet report, sending...
I20260806 01:08:29.593799 5658 ts_manager.cc:194] Registered new tserver with Master: 5025f3788b924c4bba45b576710d94e9 (127.22.63.65:44907)
I20260806 01:08:29.595265 5658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:52983
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:08:29.646898 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:30.009730 5976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:30.010076 5976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:30.010249 5976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:30.017653 5976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:30.017804 5976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:30.017995 5976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:08:30.028599 5976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:30.030077 5976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:30.031433 5976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:30.045065 5987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:30.045112 5984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:30.045151 5982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:30.045866 5976 server_base.cc:1034] running on GCE node
I20260806 01:08:30.046546 5976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:30.047474 5976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:30.048686 5976 hybrid_clock.cc:648] HybridClock initialized: now 1785978510048609 us; error 69 us; skew 500 ppm
I20260806 01:08:30.049149 5976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:30 dist-test-slave-7f01 krb5kdc[5575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978510, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:30.052579 5976 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:08:30.054126 5976 webserver.cc:460] Webserver started at http://127.22.63.66:35309/ using document root <none> and password file <none>
I20260806 01:08:30.054847 5976 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:30.055001 5976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:30.055308 5976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:30.058341 5976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "32e430f549e04f128b0935b3d659fdc3"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "e5bf2e0b75413dfcc95860577abd2b2b"
server_key_iv: "2b691c908a5a7c56fda43b8894087569"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.059144 5976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "32e430f549e04f128b0935b3d659fdc3"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "e5bf2e0b75413dfcc95860577abd2b2b"
server_key_iv: "2b691c908a5a7c56fda43b8894087569"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.064399 5976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 01:08:30.068022 6012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.069343 5976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:08:30.069540 5976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "32e430f549e04f128b0935b3d659fdc3"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "e5bf2e0b75413dfcc95860577abd2b2b"
server_key_iv: "2b691c908a5a7c56fda43b8894087569"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.069748 5976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:30.085618 5976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:30.090046 5976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:30.090368 5976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:30.091393 5976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:30.092859 5976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:30.092971 5976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.093132 5976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:30.093209 5976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.127701 5976 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:37849
I20260806 01:08:30.127815 6186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:37849 every 8 connection(s)
I20260806 01:08:30.129048 5976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:08:30.137105 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 5976
I20260806 01:08:30.137306 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:08:30.137779 22781 external_mini_cluster.cc:1428] Setting key cf9504215f6b17d6e3724a7d50970101
Aug 06 01:08:30 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978510, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:30.149335 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.131764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:56745 (local address 127.22.63.126:36729)
0806 01:08:30.132269 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.132273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.133140 (+ 867us) server_negotiation.cc:407] Connection header received
0806 01:08:30.134944 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.134950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.135043 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.135169 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:30.136751 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.137594 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.139306 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.139931 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.140083 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.143508 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:30.143513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:30.143517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:30.143549 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:30.146178 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.146930 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.146936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.146944 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.147086 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.147538 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.147545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.147548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.147879 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:30.148048 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.148723 (+ 675us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.148946 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":283,"threads_started":1}
I20260806 01:08:30.150312 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.132252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36729 (local address 127.22.63.66:56745)
0806 01:08:30.132965 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.133006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.134671 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.135414 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.135422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.135997 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:30.136568 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.136584 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.137748 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.137758 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.139158 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.139170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.140172 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.140181 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.140424 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.141140 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:30.141162 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:30.143279 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:30.146334 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:30.146343 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:30.146359 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:30.146795 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:30.147208 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:30.147213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:30.147217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:30.147413 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:30.148012 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:30.148020 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:30.148354 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.148632 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.149154 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":281,"threads_started":1}
I20260806 01:08:30.151926 6188 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36729
I20260806 01:08:30.152227 6188 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:30.152972 6188 heartbeater.cc:511] Master 127.22.63.126:36729 requested a full tablet report, sending...
I20260806 01:08:30.154562 5658 ts_manager.cc:194] Registered new tserver with Master: 32e430f549e04f128b0935b3d659fdc3 (127.22.63.66:37849)
I20260806 01:08:30.155887 5658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:56745
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:08:30.204608 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:08:30.550997 6195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:08:30.551366 6195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:08:30.551554 6195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:08:30.559163 6195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:08:30.559353 6195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:08:30.559536 6195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:08:30.569913 6195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:36729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:08:30.571203 6195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:08:30.572664 6195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:08:30.586422 6203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:30.586527 6208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:08:30.586549 6201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:08:30.587219 6195 server_base.cc:1034] running on GCE node
I20260806 01:08:30.587867 6195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:08:30.588857 6195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:08:30.590062 6195 hybrid_clock.cc:648] HybridClock initialized: now 1785978510590014 us; error 38 us; skew 500 ppm
I20260806 01:08:30.590394 6195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:08:30 dist-test-slave-7f01 krb5kdc[5575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978510, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:08:30.598270 5965 heartbeater.cc:503] Master 127.22.63.126:36729 was elected leader, sending a full tablet report...
I20260806 01:08:30.598703 6195 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:08:30.600546 6195 webserver.cc:460] Webserver started at http://127.22.63.67:38265/ using document root <none> and password file <none>
I20260806 01:08:30.601308 6195 fs_manager.cc:359] Metadata directory not provided
I20260806 01:08:30.601459 6195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:08:30.601773 6195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:08:30.604853 6195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "7676c542b72e4ca29e2e68f32c2dfd71"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "83b90498764aaa23664b465d1a09aa92"
server_key_iv: "b04f1c25adf9208f47d5a90ff3329347"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.605614 6195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "7676c542b72e4ca29e2e68f32c2dfd71"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "83b90498764aaa23664b465d1a09aa92"
server_key_iv: "b04f1c25adf9208f47d5a90ff3329347"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.610838 6195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:08:30.614574 6217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.615993 6195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:08:30.616165 6195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "7676c542b72e4ca29e2e68f32c2dfd71"
format_stamp: "Formatted at 2026-08-06 01:08:30 on dist-test-slave-7f01"
server_key: "83b90498764aaa23664b465d1a09aa92"
server_key_iv: "b04f1c25adf9208f47d5a90ff3329347"
server_key_version: "encryptionkey@0"
I20260806 01:08:30.616369 6195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:08:30.651501 6195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:08:30.656301 6195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:08:30.656626 6195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:08:30.657653 6195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:08:30.659086 6195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:08:30.659198 6195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.659356 6195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:08:30.659435 6195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:08:30.694645 6195 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:44521
I20260806 01:08:30.694677 6392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:44521 every 8 connection(s)
I20260806 01:08:30.696101 6195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:08:30.705080 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 6195
I20260806 01:08:30.705279 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:08:30.705755 22781 external_mini_cluster.cc:1428] Setting key a9932eb25c6080094c616c77302380b8
Aug 06 01:08:30 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978510, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:08:30.717330 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.698848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:56395 (local address 127.22.63.126:36729)
0806 01:08:30.699361 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.699366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.700243 (+ 877us) server_negotiation.cc:407] Connection header received
0806 01:08:30.702060 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.702066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.702158 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.702281 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:30.703846 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.704617 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.706604 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.707171 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.707284 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.711138 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:30.711143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:30.711147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:30.711184 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:30.714040 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.714810 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.714816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.714819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.714958 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.715412 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.715422 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.715425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.715733 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:30.715971 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.716695 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.716923 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":271,"threads_started":1}
I20260806 01:08:30.718355 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.699338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:36729 (local address 127.22.63.67:56395)
0806 01:08:30.700083 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.700121 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.701778 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.702529 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.702537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.703046 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:08:30.703637 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.703655 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.704831 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.704835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.706440 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.706459 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.707428 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.707433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.707645 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.708476 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:08:30.708506 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:08:30.710905 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:08:30.714197 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:30.714207 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:30.714225 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:30.714673 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:30.715080 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:08:30.715085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:08:30.715089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:08:30.715284 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:08:30.715921 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:08:30.715932 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:08:30.716309 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.716593 (+ 284us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.717151 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":271,"threads_started":1}
I20260806 01:08:30.719952 6393 heartbeater.cc:347] Connected to a master server at 127.22.63.126:36729
I20260806 01:08:30.720366 6393 heartbeater.cc:464] Registering TS with master...
I20260806 01:08:30.721367 6393 heartbeater.cc:511] Master 127.22.63.126:36729 requested a full tablet report, sending...
I20260806 01:08:30.722913 5658 ts_manager.cc:194] Registered new tserver with Master: 7676c542b72e4ca29e2e68f32c2dfd71 (127.22.63.67:44521)
I20260806 01:08:30.723827 5658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:56395
I20260806 01:08:30.734032 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:08:30.751535 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.738910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.22.63.126:36729)
0806 01:08:30.739113 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.739118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.739601 (+ 483us) server_negotiation.cc:407] Connection header received
0806 01:08:30.739845 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.739850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.739922 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.740014 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:30.741451 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.742222 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.743457 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.743907 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.744007 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.746401 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:30.746406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:30.746409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:30.746441 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:30.749152 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.749803 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.749806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.749809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.749954 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:30.750291 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:30.750294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:30.750295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:30.750617 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:30.750794 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.751131 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.751304 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:08:30.755728 5658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:08:30.762683 5658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:08:30.793051 6412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.775033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:44521 (local address 127.0.0.1:59058)
0806 01:08:30.776573 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.776625 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.776792 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.781646 (+ 4854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.781654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.781718 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:08:30.782267 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.782285 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.785965 (+ 3680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.785975 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.789212 (+ 3237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.789240 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.791632 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.791641 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.791985 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.792065 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.792204 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.792435 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"mutex_wait_us":1,"thread_start_us":1137,"threads_started":1}
I20260806 01:08:30.793880 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.776122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.22.63.66:37849)
0806 01:08:30.777309 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.777321 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.779322 (+ 2001us) server_negotiation.cc:407] Connection header received
0806 01:08:30.780374 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.780384 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.780609 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.780823 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:08:30.785981 (+ 5158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.787687 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.790913 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.792519 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.792639 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.792849 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.793504 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.793641 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1006,"thread_start_us":866,"threads_started":1}
I20260806 01:08:30.794399 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.775514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.22.63.67:44521)
0806 01:08:30.778627 (+ 3113us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.778639 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.778684 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:08:30.778813 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.778838 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.779136 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.783154 (+ 4018us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:08:30.783546 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.785735 (+ 2189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.789508 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.791423 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.793594 (+ 2171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.793836 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.793905 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.794071 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2942,"thread_start_us":2923,"threads_started":1}
I20260806 01:08:30.794644 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.775173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:37849 (local address 127.0.0.1:34070)
0806 01:08:30.777924 (+ 2751us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.777945 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.778091 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.780941 (+ 2850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.780954 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.781259 (+ 305us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:08:30.782853 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.782869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.787932 (+ 5063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.787939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.790677 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.790699 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.793066 (+ 2367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.793070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.793278 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.793307 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.794164 (+ 857us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.794283 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2581,"mutex_wait_us":1,"thread_start_us":2357,"threads_started":1}
I20260806 01:08:30.798841 6296 tablet_service.cc:1467] Processing CreateTablet for tablet caa564747f534644b4bc7f35236af0f4 (DEFAULT_TABLE table=test-table [id=bd251ac432b447d3b28eab7eaf127830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:08:30.800519 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa564747f534644b4bc7f35236af0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:30.801436 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.777846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44907 (local address 127.0.0.1:41420)
0806 01:08:30.781464 (+ 3618us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.781489 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.781618 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.791717 (+ 10099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.791722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.791780 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:08:30.792312 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.792334 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.794666 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.794673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.798136 (+ 3463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.798156 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.800437 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.800447 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.800736 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.800770 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.800858 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.801052 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3499,"mutex_wait_us":1,"thread_start_us":7706,"threads_started":1}
I20260806 01:08:30.801191 6085 tablet_service.cc:1467] Processing CreateTablet for tablet caa564747f534644b4bc7f35236af0f4 (DEFAULT_TABLE table=test-table [id=bd251ac432b447d3b28eab7eaf127830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:08:30.802616 6085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa564747f534644b4bc7f35236af0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:30.803954 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.778244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.22.63.65:44907)
0806 01:08:30.790749 (+ 12505us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.790759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.790794 (+ 35us) server_negotiation.cc:407] Connection header received
0806 01:08:30.790920 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.790929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.791216 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.792526 (+ 1310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:08:30.792812 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.794348 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.798638 (+ 4290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.800173 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.800331 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.800566 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.801765 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.802326 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12284,"thread_start_us":13154,"threads_started":1}
I20260806 01:08:30.808269 5855 tablet_service.cc:1467] Processing CreateTablet for tablet caa564747f534644b4bc7f35236af0f4 (DEFAULT_TABLE table=test-table [id=bd251ac432b447d3b28eab7eaf127830]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:08:30.809903 5855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa564747f534644b4bc7f35236af0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:08:30.819597 6422 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Bootstrap starting.
I20260806 01:08:30.823139 6423 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Bootstrap starting.
I20260806 01:08:30.824697 6422 tablet_bootstrap.cc:654] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:30.826216 6422 log.cc:824] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:30.827479 6423 tablet_bootstrap.cc:654] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:30.829528 6423 log.cc:824] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:30.832211 6422 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: No bootstrap required, opened a new log
I20260806 01:08:30.832608 6422 ts_tablet_manager.cc:1397] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.003s
I20260806 01:08:30.838549 6423 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: No bootstrap required, opened a new log
I20260806 01:08:30.838853 6423 ts_tablet_manager.cc:1397] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260806 01:08:30.846592 6422 raft_consensus.cc:348] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.847211 6422 raft_consensus.cc:374] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:30.847340 6422 raft_consensus.cc:729] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e430f549e04f128b0935b3d659fdc3, State: Initialized, Role: FOLLOWER
I20260806 01:08:30.847534 6428 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Bootstrap starting.
I20260806 01:08:30.847287 6423 raft_consensus.cc:348] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.847733 6423 raft_consensus.cc:374] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:30.847877 6423 raft_consensus.cc:729] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7676c542b72e4ca29e2e68f32c2dfd71, State: Initialized, Role: FOLLOWER
I20260806 01:08:30.848130 6422 consensus_queue.cc:260] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.848564 6423 consensus_queue.cc:260] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.849524 6188 heartbeater.cc:503] Master 127.22.63.126:36729 was elected leader, sending a full tablet report...
I20260806 01:08:30.850260 6422 ts_tablet_manager.cc:1428] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260806 01:08:30.852051 6393 heartbeater.cc:503] Master 127.22.63.126:36729 was elected leader, sending a full tablet report...
I20260806 01:08:30.852795 6423 ts_tablet_manager.cc:1428] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260806 01:08:30.854492 6428 tablet_bootstrap.cc:654] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Neither blocks nor log segments found. Creating new log.
I20260806 01:08:30.856092 6428 log.cc:824] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:08:30.859187 6428 tablet_bootstrap.cc:492] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: No bootstrap required, opened a new log
I20260806 01:08:30.859441 6428 ts_tablet_manager.cc:1397] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.005s
I20260806 01:08:30.866784 6428 raft_consensus.cc:348] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.867139 6428 raft_consensus.cc:374] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:08:30.867312 6428 raft_consensus.cc:729] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5025f3788b924c4bba45b576710d94e9, State: Initialized, Role: FOLLOWER
I20260806 01:08:30.867955 6428 consensus_queue.cc:260] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.869426 6428 ts_tablet_manager.cc:1428] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
W20260806 01:08:30.881126 6189 tablet.cc:2369] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:08:30.887563 6431 raft_consensus.cc:482] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:08:30.887904 6431 raft_consensus.cc:504] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.889935 6431 leader_election.cc:288] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7676c542b72e4ca29e2e68f32c2dfd71 (127.22.63.67:44521), 5025f3788b924c4bba45b576710d94e9 (127.22.63.65:44907)
I20260806 01:08:30.897121 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.890403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:45129 (local address 127.22.63.67:44521)
0806 01:08:30.890603 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.890609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.891612 (+ 1003us) server_negotiation.cc:407] Connection header received
0806 01:08:30.891882 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.891886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.891952 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.892076 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:08:30.892624 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.893346 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.895306 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.896170 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.896369 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.896459 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.896725 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.896853 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:08:30.897410 6438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.890431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:44521 (local address 127.22.63.66:45129)
0806 01:08:30.891448 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.891480 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.891683 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.892099 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.892104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.892133 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:08:30.892467 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.892476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.893508 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.893513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.895148 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.895161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.896401 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.896408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.896560 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.896584 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.896700 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.896841 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":60,"thread_start_us":626,"threads_started":1}
I20260806 01:08:30.897703 6437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.890508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44907 (local address 127.22.63.66:56085)
0806 01:08:30.891918 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0806 01:08:30.891932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:08:30.892075 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:08:30.892426 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:08:30.892431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:08:30.892460 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:08:30.892819 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.892827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.893826 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.893831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.895252 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:08:30.895264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.896795 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.896802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:08:30.897001 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.897027 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 01:08:30.897146 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 01:08:30.897274 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1276,"thread_start_us":1047,"threads_started":1}
I20260806 01:08:30.897789 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.890828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:56085 (local address 127.22.63.65:44907)
0806 01:08:30.891077 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.891085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.892027 (+ 942us) server_negotiation.cc:407] Connection header received
0806 01:08:30.892224 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.892227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.892286 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.892399 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:08:30.892979 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.893677 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.895422 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.896509 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.896754 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.896841 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.897189 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.897523 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:08:30.898578 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa564747f534644b4bc7f35236af0f4" candidate_uuid: "32e430f549e04f128b0935b3d659fdc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" is_pre_election: true
I20260806 01:08:30.898661 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa564747f534644b4bc7f35236af0f4" candidate_uuid: "32e430f549e04f128b0935b3d659fdc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5025f3788b924c4bba45b576710d94e9" is_pre_election: true
I20260806 01:08:30.899158 6330 raft_consensus.cc:2454] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e430f549e04f128b0935b3d659fdc3 in term 0.
I20260806 01:08:30.899211 5893 raft_consensus.cc:2454] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e430f549e04f128b0935b3d659fdc3 in term 0.
I20260806 01:08:30.899922 6013 leader_election.cc:302] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32e430f549e04f128b0935b3d659fdc3, 5025f3788b924c4bba45b576710d94e9, 7676c542b72e4ca29e2e68f32c2dfd71; no voters:
I20260806 01:08:30.900401 6431 raft_consensus.cc:2781] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:08:30.900578 6431 raft_consensus.cc:482] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:08:30.900712 6431 raft_consensus.cc:3037] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:30.903151 6431 raft_consensus.cc:504] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.903988 6431 leader_election.cc:288] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [CANDIDATE]: Term 1 election: Requested vote from peers 7676c542b72e4ca29e2e68f32c2dfd71 (127.22.63.67:44521), 5025f3788b924c4bba45b576710d94e9 (127.22.63.65:44907)
I20260806 01:08:30.904381 6330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa564747f534644b4bc7f35236af0f4" candidate_uuid: "32e430f549e04f128b0935b3d659fdc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7676c542b72e4ca29e2e68f32c2dfd71"
I20260806 01:08:30.904495 5893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa564747f534644b4bc7f35236af0f4" candidate_uuid: "32e430f549e04f128b0935b3d659fdc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5025f3788b924c4bba45b576710d94e9"
I20260806 01:08:30.904601 6330 raft_consensus.cc:3037] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:30.904721 5893 raft_consensus.cc:3037] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:08:30.907004 6330 raft_consensus.cc:2454] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e430f549e04f128b0935b3d659fdc3 in term 1.
I20260806 01:08:30.907460 6017 leader_election.cc:302] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e430f549e04f128b0935b3d659fdc3, 7676c542b72e4ca29e2e68f32c2dfd71; no voters:
I20260806 01:08:30.907627 5893 raft_consensus.cc:2454] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e430f549e04f128b0935b3d659fdc3 in term 1.
I20260806 01:08:30.908047 6431 raft_consensus.cc:2781] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:08:30.908540 6431 raft_consensus.cc:686] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 1 LEADER]: Becoming Leader. State: Replica: 32e430f549e04f128b0935b3d659fdc3, State: Running, Role: LEADER
I20260806 01:08:30.909137 6431 consensus_queue.cc:237] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } }
I20260806 01:08:30.914213 5658 catalog_manager.cc:5495] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32e430f549e04f128b0935b3d659fdc3 (127.22.63.66). New cstate: current_term: 1 leader_uuid: "32e430f549e04f128b0935b3d659fdc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e430f549e04f128b0935b3d659fdc3" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 } health_report { overall_health: UNKNOWN } } }
I20260806 01:08:30.947721 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.942217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.22.63.66:37849)
0806 01:08:30.942465 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.942470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:30.942489 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:08:30.942660 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:30.942664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:30.942745 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:30.942897 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:08:30.943305 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.944164 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.945489 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:30.945934 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:30.946039 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:30.946500 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:08:30.947049 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:08:30.947178 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:30.947318 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 01:08:30.947390 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 01:08:30.947911 6394 tablet.cc:2369] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:08:30.956314 6330 raft_consensus.cc:1261] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 1 FOLLOWER]: Refusing update from remote peer 32e430f549e04f128b0935b3d659fdc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:08:30.956313 5893 raft_consensus.cc:1261] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 1 FOLLOWER]: Refusing update from remote peer 32e430f549e04f128b0935b3d659fdc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:08:30.957449 6431 consensus_queue.cc:1035] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7676c542b72e4ca29e2e68f32c2dfd71" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:08:30.957809 6440 consensus_queue.cc:1035] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5025f3788b924c4bba45b576710d94e9" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:08:30.970629 6443 mvcc.cc:205] Tried to move back new op lower bound from 7315367980865908736 to 7315367980688265216. Current Snapshot: MvccSnapshot[applied={T|T < 7315367980865908736}]
I20260806 01:08:30.986409 5658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:08:30.986713 5658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:08:30.990607 5658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2785cb39b51d4d2d95d7a964f46591eb: Sending DeleteTablet for 3 replicas of tablet caa564747f534644b4bc7f35236af0f4
I20260806 01:08:30.991927 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet caa564747f534644b4bc7f35236af0f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:08:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:59058
I20260806 01:08:30.992063 5855 tablet_service.cc:1514] Processing DeleteTablet for tablet caa564747f534644b4bc7f35236af0f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:08:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:41420
I20260806 01:08:30.992152 6085 tablet_service.cc:1514] Processing DeleteTablet for tablet caa564747f534644b4bc7f35236af0f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:08:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:34070
I20260806 01:08:30.994331 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 5751
I20260806 01:08:31.023165 6457 tablet_replica.cc:330] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: stopping tablet replica
I20260806 01:08:31.023902 6457 raft_consensus.cc:2229] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:08:31.024657 6457 raft_consensus.cc:2258] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:08:31.032963 6457 ts_tablet_manager.cc:1905] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:08:31.038731 6455 tablet_replica.cc:330] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: stopping tablet replica
I20260806 01:08:31.039371 6455 raft_consensus.cc:2229] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:08:31.040091 6455 raft_consensus.cc:2258] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:08:31.043009 6455 ts_tablet_manager.cc:1905] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:08:31.044281 6459 tablet_replica.cc:330] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: stopping tablet replica
I20260806 01:08:31.044881 6459 raft_consensus.cc:2229] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:08:31.045744 6459 raft_consensus.cc:2258] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:08:31.048794 6459 ts_tablet_manager.cc:1905] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:08:31.051695 6457 ts_tablet_manager.cc:1918] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:08:31.051939 6457 log.cc:1196] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/caa564747f534644b4bc7f35236af0f4
I20260806 01:08:31.052510 6457 ts_tablet_manager.cc:1939] T caa564747f534644b4bc7f35236af0f4 P 5025f3788b924c4bba45b576710d94e9: Deleting consensus metadata
I20260806 01:08:31.054024 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:30.996085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41436 (local address 127.22.63.65:44907)
0806 01:08:30.996337 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:30.996346 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:31.019653 (+ 23307us) server_negotiation.cc:407] Connection header received
0806 01:08:31.020380 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:31.020391 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:31.020528 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:31.020621 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:31.025857 (+ 5236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.027447 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.035906 (+ 8459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.036839 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.036977 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:31.045167 (+ 8190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:31.045187 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:31.045214 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:31.045280 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:31.049616 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.050542 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.050551 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.050555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.050727 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.051159 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.051165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.051167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.051684 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:31.052049 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:31.053346 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0806 01:08:31.053597 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:08:31.054728 5958 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:31.055107 5632 catalog_manager.cc:4841] TS 5025f3788b924c4bba45b576710d94e9 (127.22.63.65:44907): tablet caa564747f534644b4bc7f35236af0f4 (table test-table [id=bd251ac432b447d3b28eab7eaf127830]) successfully deleted
I20260806 01:08:31.057943 6459 ts_tablet_manager.cc:1918] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:08:31.058241 6459 log.cc:1196] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/caa564747f534644b4bc7f35236af0f4
I20260806 01:08:31.058954 6459 ts_tablet_manager.cc:1939] T caa564747f534644b4bc7f35236af0f4 P 32e430f549e04f128b0935b3d659fdc3: Deleting consensus metadata
I20260806 01:08:31.060351 6455 ts_tablet_manager.cc:1918] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:08:31.060570 6455 log.cc:1196] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/caa564747f534644b4bc7f35236af0f4
I20260806 01:08:31.061113 5632 catalog_manager.cc:4841] TS 32e430f549e04f128b0935b3d659fdc3 (127.22.63.66:37849): tablet caa564747f534644b4bc7f35236af0f4 (table test-table [id=bd251ac432b447d3b28eab7eaf127830]) successfully deleted
I20260806 01:08:31.061210 6455 ts_tablet_manager.cc:1939] T caa564747f534644b4bc7f35236af0f4 P 7676c542b72e4ca29e2e68f32c2dfd71: Deleting consensus metadata
I20260806 01:08:31.063405 5636 catalog_manager.cc:4841] TS 7676c542b72e4ca29e2e68f32c2dfd71 (127.22.63.67:44521): tablet caa564747f534644b4bc7f35236af0f4 (table test-table [id=bd251ac432b447d3b28eab7eaf127830]) successfully deleted
I20260806 01:08:31.155224 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 5751
I20260806 01:08:31.181391 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 5976
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:08:31.203161 6418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:31.182231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.22.63.66:37849)
0806 01:08:31.182458 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:31.182465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:31.182491 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:08:31.182650 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:31.182656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:31.182753 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:31.182878 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:31.184219 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.185647 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.186898 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.187671 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.187870 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:31.195005 (+ 7135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:31.195033 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:31.195058 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:31.195133 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:31.199630 (+ 4497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.200729 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.200745 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.200751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.200939 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.201320 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.201325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.201329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.201901 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:31.202239 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:31.202422 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 01:08:31.202703 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:08:31.204010 6178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:31.319105 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 5976
I20260806 01:08:31.346176 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 6195
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[5575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:08:31.362255 6416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:08:31.347024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.22.63.67:44521)
0806 01:08:31.347222 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:08:31.347228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:08:31.347328 (+ 100us) server_negotiation.cc:407] Connection header received
0806 01:08:31.347562 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:08:31.347567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:08:31.347634 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:08:31.347757 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:08:31.349108 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.349848 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.351165 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:08:31.351587 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:08:31.351734 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:08:31.356515 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:08:31.356531 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:08:31.356547 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:08:31.356588 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:08:31.359230 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.360076 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.360090 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.360093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.360237 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:08:31.360651 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:08:31.360661 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:08:31.360663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:08:31.361029 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:08:31.361347 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 01:08:31.361684 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 01:08:31.361919 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:08:31.362922 6388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:31.453547 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 6195
I20260806 01:08:31.478475 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 5601
I20260806 01:08:31.479835 5699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:08:31.592391 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 5601
2026-08-06T01:08:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4843 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[6476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[6476](info): set up 2 sockets
Aug 06 01:08:31 dist-test-slave-7f01 krb5kdc[6476](info): commencing operation
krb5kdc: starting...
W20260806 01:08:32.945097 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:08:33 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:08:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangeradmin_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangeradmin_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangerlookup_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangerlookup_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
I20260806 01:08:33.372740 22781 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfkFK3E/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/postgres -l logfile start
2026-08-06 01:08:36.046 UTC [6529] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:08:36.046 UTC [6529] LOG: listening on IPv4 address "127.22.63.84", port 34563
2026-08-06 01:08:36.050 UTC [6529] LOG: listening on Unix socket "/tmp/.s.PGSQL.34563"
2026-08-06 01:08:36.072 UTC [6533] LOG: database system was shut down at 2026-08-06 01:08:34 UTC
2026-08-06 01:08:36.076 UTC [6529] LOG: database system is ready to accept connections
I20260806 01:08:37.254971 22781 mini_postgres.cc:96] Postgres bound to 34563
2026-08-06 01:08:37.281 UTC [6549] FATAL: database "slave" does not exist
127.22.63.84:34563 - accepting connections
I20260806 01:08:37.282308 22781 mini_ranger.cc:161] Starting Ranger...
I20260806 01:08:37.317418 22781 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:08:37.860425 22781 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:08:37.860630 22781 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin
2026-08-06 01:08:38,187 [I] DB FLAVOR :POSTGRES
2026-08-06 01:08:38,187 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:08:38,187 [I] Checking connection..
2026-08-06 01:08:38,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:08:38,458 [I] Checking connection passed.
2026-08-06 01:08:38,458 [I] --------- Verifying version history table ---------
2026-08-06 01:08:38,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:08:38,726 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:08:38,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:08:38,995 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:08:38,995 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:08:38,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:08:39.224 UTC [6646] WARNING: there is no transaction in progress
2026-08-06 01:08:39,232 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:08:39,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:08:39,493 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:08:39,493 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:08:39,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:08:39,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:08:40,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-06 01:08:40,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:08:40,530 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:08:40,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:08:40,791 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:08:40,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:08:41,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:08:41,297 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:08:41,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:08:42.308 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.325 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.342 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.358 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.376 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.394 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.450 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.457 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.472 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.482 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.492 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.506 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.515 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.522 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.528 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:42.535 UTC [6833] WARNING: there is no transaction in progress
2026-08-06 01:08:43,176 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:08:43,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:08:43,438 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:08:43,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:08:43,709 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:08:43,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:08:43,973 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:08:43,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:08:44,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:08:44,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-06 01:08:44,722 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:08:44,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:08:44,984 [I] DB_PATCHES have already been applied
I20260806 01:08:44.990394 22781 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfkFK3E/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:08:44.990548 22781 mini_ranger.cc:193] Using host: 127.22.63.84
I20260806 01:08:45.014182 22781 mini_ranger.cc:227] Ranger admin URL: http://127.22.63.84:33737
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:08:46.372705 22781 mini_ranger.cc:160] Time spent starting Ranger: real 9.090s user 0.000s sys 0.071s
Aug 06, 2026 1:09:06 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:09:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 1:09:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:09:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:09:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:09:08.179865 22781 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangerkms_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/rangerkms_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:09:08 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978548, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:09:08.401794 22781 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:09:08.431027 22781 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:09:09.288301 22781 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:09:09.288465 22781 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms
I20260806 01:09:09.288633 22781 mini_ranger_kms.cc:227] Using postgres at 127.22.63.84:34563
2026-08-06 01:09:09,866 [I] DB FLAVOR :POSTGRES
2026-08-06 01:09:09,866 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:09:09,867 [I] Checking connection
2026-08-06 01:09:09,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:09:10,104 [I] connection success
2026-08-06 01:09:10,105 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:09:10,105 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:09:10,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:09:10,336 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:09:10,336 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:09:10,337 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:09:10,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:34563/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:09:10,619 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:09:11.095476 22781 mini_ranger_kms.cc:310] Created kms service
I20260806 01:09:11.333400 22781 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:09:11.382660 22781 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:09:11 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978548, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:09:11.777256 22781 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:09:11.777686 22781 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:09:11.777738 22781 mini_ranger_kms.cc:240] Using host: 127.22.63.84
I20260806 01:09:11.806378 22781 mini_ranger_kms.cc:276] Ranger KMS PID: 7136
I20260806 01:09:11.806587 22781 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.63.84:54471
Aug 06, 2026 1:09:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:09:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:09:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:09:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:09:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:09:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:09:14 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:09:14.607393 22781 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.206s user 0.125s sys 0.500s
I20260806 01:09:14.608076 22781 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:09:14.608319 22781 mini_ranger_kms.cc:401] 127.22.63.84:54471/kms/v1/keys
Aug 06 01:09:14 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Aug 06 01:09:15 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Aug 06 01:09:17 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:09:17.443840 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45119
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45119
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:54471/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:09:17.809736 7214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:09:17.810034 7214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:09:17.810177 7214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:09:17.817755 7214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:09:17.817878 7214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:09:17.817957 7214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:09:17.818053 7214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:09:17.818133 7214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:09:17.828938 7214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.63.84:54471/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45119
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:09:17.830344 7214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:09:17.831802 7214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:09:17.846388 7226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:17.846342 7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:17.847559 7214 server_base.cc:1034] running on GCE node
W20260806 01:09:17.848006 7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:17.848901 7214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:09:17.850457 7214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:09:17.851682 7214 hybrid_clock.cc:648] HybridClock initialized: now 1785978557851630 us; error 48 us; skew 500 ppm
I20260806 01:09:17.852087 7214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:09:17 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:09:17.858457 7214 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:09:17.860198 7214 webserver.cc:460] Webserver started at http://127.22.63.126:42651/ using document root <none> and password file <none>
I20260806 01:09:17.860898 7214 fs_manager.cc:359] Metadata directory not provided
I20260806 01:09:17.861038 7214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:09:17.861358 7214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:09:17 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:09:17.928076 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "6b2869a152524d6fb511eea9f7a5ff3a"
format_stamp: "Formatted at 2026-08-06 01:09:17 on dist-test-slave-7f01"
server_key: "7dc50a5d344c2c4fe3b732e4bab50912"
server_key_iv: "ca230fab101b050f1419f0e2db2dd55d"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:17.929064 7214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "6b2869a152524d6fb511eea9f7a5ff3a"
format_stamp: "Formatted at 2026-08-06 01:09:17 on dist-test-slave-7f01"
server_key: "7dc50a5d344c2c4fe3b732e4bab50912"
server_key_iv: "ca230fab101b050f1419f0e2db2dd55d"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:17.934702 7214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:09:17.955261 7239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:17.956843 7214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:09:17.957029 7214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "6b2869a152524d6fb511eea9f7a5ff3a"
format_stamp: "Formatted at 2026-08-06 01:09:17 on dist-test-slave-7f01"
server_key: "7dc50a5d344c2c4fe3b732e4bab50912"
server_key_iv: "ca230fab101b050f1419f0e2db2dd55d"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:17.957273 7214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:09:17.998127 7214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:09:18.002557 7214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:09:18.002828 7214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:09:18.021727 7214 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:45119
I20260806 01:09:18.021760 7322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:45119 every 8 connection(s)
I20260806 01:09:18.023087 7214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:09:18.026953 7324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:09:18.030514 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 7214
I20260806 01:09:18.030743 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:09:18.031611 22781 external_mini_cluster.cc:1428] Setting key 57ef20771e660665c99d18ce909f2338
I20260806 01:09:18.040688 7324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: Bootstrap starting.
I20260806 01:09:18.045002 7324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: Neither blocks nor log segments found. Creating new log.
I20260806 01:09:18.046443 7324 log.cc:824] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:09:18 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:09:18.049787 7324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: No bootstrap required, opened a new log
I20260806 01:09:18.055603 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:18.033858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.22.63.126:45119)
0806 01:09:18.034667 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:18.034678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:18.034720 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:09:18.036442 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:18.036466 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:18.036851 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:18.037195 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:18.038856 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:18.040217 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:18.042092 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:18.042790 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:18.042936 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:18.048263 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:18.048272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:18.048287 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:18.048329 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:18.051422 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:18.052221 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:18.052228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:18.052234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:18.052385 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:18.052785 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:18.052790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:18.052793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:18.053346 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:18.053666 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:18.054158 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 01:09:18.054564 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":393,"threads_started":1}
I20260806 01:09:18.057844 7324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } }
I20260806 01:09:18.058250 7324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:09:18.058387 7324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2869a152524d6fb511eea9f7a5ff3a, State: Initialized, Role: FOLLOWER
I20260806 01:09:18.058941 7324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } }
I20260806 01:09:18.059150 7324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:09:18.059247 7324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:09:18.059373 7324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:09:18.061625 7324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } }
I20260806 01:09:18.062216 7324 leader_election.cc:302] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b2869a152524d6fb511eea9f7a5ff3a; no voters:
I20260806 01:09:18.062887 7324 leader_election.cc:288] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:09:18.063036 7330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:09:18.063829 7330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [term 1 LEADER]: Becoming Leader. State: Replica: 6b2869a152524d6fb511eea9f7a5ff3a, State: Running, Role: LEADER
I20260806 01:09:18.064399 7330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } }
I20260806 01:09:18.065232 7324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:09:18.067397 7332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b2869a152524d6fb511eea9f7a5ff3a. Latest consensus state: current_term: 1 leader_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } } }
I20260806 01:09:18.067801 7332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [sys.catalog]: This master's current role is: LEADER
I20260806 01:09:18.067508 7331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2869a152524d6fb511eea9f7a5ff3a" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45119 } } }
I20260806 01:09:18.067991 7331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a [sys.catalog]: This master's current role is: LEADER
I20260806 01:09:18.072885 7339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:09:18.078882 7339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:09:18.079965 7324 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:09:18.098311 7339 catalog_manager.cc:1347] Generated new cluster ID: 825dd23ad5b9497baa21e649a6b06042
I20260806 01:09:18.098560 7339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:09:18.112038 7339 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:09:18.114135 7339 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:09:18.124429 7339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: Generated new TSK 0
I20260806 01:09:18.125672 7339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:09:18.932534 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:54471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:09:19 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:09:19.342922 7405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:09:19.343271 7405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:09:19.343453 7405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:09:19.352054 7405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:09:19.352209 7405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:09:19.352403 7405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:09:19.364032 7405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.63.84:54471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:09:19.365857 7405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:09:19.367393 7405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:09:19.383773 7424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:19.383889 7426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:19.384457 7430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:19.389853 7405 server_base.cc:1034] running on GCE node
I20260806 01:09:19.390722 7405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:09:19.392055 7405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:09:19.393322 7405 hybrid_clock.cc:648] HybridClock initialized: now 1785978559393186 us; error 123 us; skew 500 ppm
I20260806 01:09:19.393731 7405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:09:19 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:09:19.397611 7405 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:09:19.399583 7405 webserver.cc:460] Webserver started at http://127.22.63.65:46069/ using document root <none> and password file <none>
I20260806 01:09:19.400382 7405 fs_manager.cc:359] Metadata directory not provided
I20260806 01:09:19.400542 7405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:09:19.400969 7405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:09:19 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:09:19.428915 7405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "81a049e8e86840faa3a27b6d389f03c8"
format_stamp: "Formatted at 2026-08-06 01:09:19 on dist-test-slave-7f01"
server_key: "602bd849c8cec24fbaf5fd1525cae8a9"
server_key_iv: "27f9afae6d533422c8da4e9c0c2841f7"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:19.430394 7405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "81a049e8e86840faa3a27b6d389f03c8"
format_stamp: "Formatted at 2026-08-06 01:09:19 on dist-test-slave-7f01"
server_key: "602bd849c8cec24fbaf5fd1525cae8a9"
server_key_iv: "27f9afae6d533422c8da4e9c0c2841f7"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:19.441320 7405 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.000s
I20260806 01:09:19.480259 7441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:19.482367 7405 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.009s sys 0.001s
I20260806 01:09:19.482661 7405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "81a049e8e86840faa3a27b6d389f03c8"
format_stamp: "Formatted at 2026-08-06 01:09:19 on dist-test-slave-7f01"
server_key: "602bd849c8cec24fbaf5fd1525cae8a9"
server_key_iv: "27f9afae6d533422c8da4e9c0c2841f7"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:19.483057 7405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:09:19.540870 7405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:09:19.546639 7405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:09:19.547049 7405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:09:19.548928 7405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:09:19.550627 7405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:09:19.550745 7405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:19.550895 7405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:09:19.550985 7405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:19.614154 7405 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:41235
I20260806 01:09:19.614197 7623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:41235 every 8 connection(s)
I20260806 01:09:19.616086 7405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:09:19.619711 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 7405
I20260806 01:09:19.619962 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:09:19.620741 22781 external_mini_cluster.cc:1428] Setting key 4a01f263e2e4e86590dfd73f0fe0c283
Aug 06 01:09:19 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
I20260806 01:09:19.696010 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:19.641589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:51577 (local address 127.22.63.126:45119)
0806 01:09:19.663380 (+ 21791us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:19.663387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:19.665406 (+ 2019us) server_negotiation.cc:407] Connection header received
0806 01:09:19.674436 (+ 9030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:19.674442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:19.674555 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:19.674650 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:19.678369 (+ 3719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:19.679597 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:19.682269 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:19.682939 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:19.683042 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:19.688000 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:19.688006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:19.688012 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:19.688057 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:19.691972 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:19.692959 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:19.692965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:19.692969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:19.693144 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:19.693714 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:19.693719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:19.693722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:19.694096 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:19.694247 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:19.695337 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0806 01:09:19.695580 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21630,"thread_start_us":21817,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
I20260806 01:09:19.698055 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:19.640934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45119 (local address 127.22.63.65:51577)
0806 01:09:19.664235 (+ 23301us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:19.664305 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:19.673990 (+ 9685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:19.675328 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:19.675342 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:19.677209 (+ 1867us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:09:19.678124 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:19.678154 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:19.679865 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:19.679871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:19.682084 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:19.682101 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:19.683406 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:19.683413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:19.683708 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:19.684679 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:09:19.684712 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:09:19.687692 (+ 2980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:09:19.692175 (+ 4483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:19.692185 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:19.692205 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:19.692806 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:19.693285 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:19.693298 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:19.693303 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:19.693561 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:19.694328 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:09:19.694337 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:09:19.694808 (+ 471us) client_negotiation.cc:769] Sending connection context
0806 01:09:19.695184 (+ 376us) client_negotiation.cc:240] Negotiation successful
0806 01:09:19.695790 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22892,"thread_start_us":30723,"threads_started":1}
I20260806 01:09:19.699950 7624 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45119
I20260806 01:09:19.700279 7624 heartbeater.cc:464] Registering TS with master...
I20260806 01:09:19.701140 7624 heartbeater.cc:511] Master 127.22.63.126:45119 requested a full tablet report, sending...
I20260806 01:09:19.703828 7273 ts_manager.cc:194] Registered new tserver with Master: 81a049e8e86840faa3a27b6d389f03c8 (127.22.63.65:41235)
I20260806 01:09:19.705883 7273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:51577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
W20260806 01:09:19.742655 22781 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.63.66: real 0.119s user 0.005s sys 0.019s
I20260806 01:09:19.743625 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:54471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:09:19 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Aug 06 01:09:20 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978559, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
W20260806 01:09:20.347935 7635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:09:20.348568 7635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:09:20.348845 7635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:09:20.366096 7635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:09:20.366459 7635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:09:20.366945 7635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:09:20.388970 7635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.63.84:54471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:09:20.391405 7635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:09:20.393872 7635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:09:20.440613 7643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:20.441664 7642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:20.447113 7635 server_base.cc:1034] running on GCE node
W20260806 01:09:20.448007 7645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:20.449072 7635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:09:20.450549 7635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:09:20.451921 7635 hybrid_clock.cc:648] HybridClock initialized: now 1785978560451783 us; error 71 us; skew 500 ppm
I20260806 01:09:20.452589 7635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:09:20 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978560, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:09:20.458941 7635 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:09:20.463038 7635 webserver.cc:460] Webserver started at http://127.22.63.66:36451/ using document root <none> and password file <none>
I20260806 01:09:20.464494 7635 fs_manager.cc:359] Metadata directory not provided
I20260806 01:09:20.464826 7635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:09:20.465629 7635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:09:20 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978560, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:09:20.490892 7635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "78f5ed91cf9b4184a14c1965bc48a710"
format_stamp: "Formatted at 2026-08-06 01:09:20 on dist-test-slave-7f01"
server_key: "64f19efbb9fee42575dad27d9d9b3757"
server_key_iv: "04994ca336b454d37084b17685f18b01"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:20.497392 7635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "78f5ed91cf9b4184a14c1965bc48a710"
format_stamp: "Formatted at 2026-08-06 01:09:20 on dist-test-slave-7f01"
server_key: "64f19efbb9fee42575dad27d9d9b3757"
server_key_iv: "04994ca336b454d37084b17685f18b01"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:20.507937 7635 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.003s
I20260806 01:09:20.527971 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:20.530097 7635 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260806 01:09:20.530324 7635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "78f5ed91cf9b4184a14c1965bc48a710"
format_stamp: "Formatted at 2026-08-06 01:09:20 on dist-test-slave-7f01"
server_key: "64f19efbb9fee42575dad27d9d9b3757"
server_key_iv: "04994ca336b454d37084b17685f18b01"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:20.530745 7635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:09:20.559302 7635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:09:20.567006 7635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:09:20.567520 7635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:09:20.569370 7635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:09:20.572098 7635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:09:20.572228 7635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:20.572485 7635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:09:20.572626 7635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:20.675688 7635 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:41443
I20260806 01:09:20.675953 7851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:41443 every 8 connection(s)
I20260806 01:09:20.678103 7635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:09:20.680059 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 7635
I20260806 01:09:20.680280 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:09:20.680950 22781 external_mini_cluster.cc:1428] Setting key 4edbb4d193d4ce0f5ff0f857b7b11d7d
I20260806 01:09:20.709957 7624 heartbeater.cc:503] Master 127.22.63.126:45119 was elected leader, sending a full tablet report...
Aug 06 01:09:20 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978560, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
I20260806 01:09:20.737766 7857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:20.695032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:58155 (local address 127.22.63.126:45119)
0806 01:09:20.711266 (+ 16234us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:20.711273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:20.711295 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:09:20.711349 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:20.711355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:20.711510 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:20.718468 (+ 6958us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:20.718681 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:20.720134 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:20.722402 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:20.723304 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:20.724673 (+ 1369us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:20.728790 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:20.728795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:20.728798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:20.728833 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:20.733453 (+ 4620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:20.734439 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:20.734447 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:20.734453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:20.734627 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:20.735157 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:20.735165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:20.735170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:20.735666 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:20.735862 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:20.736762 (+ 900us) server_negotiation.cc:299] Negotiation successful
0806 01:09:20.737061 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16097,"thread_start_us":15979,"threads_started":1}
I20260806 01:09:20.738690 7856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:20.694145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45119 (local address 127.22.63.66:58155)
0806 01:09:20.700643 (+ 6498us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:20.700700 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:20.704466 (+ 3766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:20.716243 (+ 11777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:20.716262 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:20.717099 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:09:20.718207 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:20.718249 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:20.720372 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:20.720380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:20.722202 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:20.722216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:20.723608 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:20.723619 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:20.724090 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:20.725064 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:09:20.725096 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:09:20.728474 (+ 3378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:09:20.733641 (+ 5167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:20.733651 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:20.733670 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:20.734290 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:20.734771 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:20.734776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:20.734780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:20.735030 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:20.735861 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:09:20.735869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:09:20.736318 (+ 449us) client_negotiation.cc:769] Sending connection context
0806 01:09:20.736650 (+ 332us) client_negotiation.cc:240] Negotiation successful
0806 01:09:20.737222 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6160,"thread_start_us":5869,"threads_started":1}
I20260806 01:09:20.740466 7854 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45119
I20260806 01:09:20.740801 7854 heartbeater.cc:464] Registering TS with master...
I20260806 01:09:20.741653 7854 heartbeater.cc:511] Master 127.22.63.126:45119 requested a full tablet report, sending...
I20260806 01:09:20.743377 7273 ts_manager.cc:194] Registered new tserver with Master: 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443)
I20260806 01:09:20.744323 7273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:58155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:09:20.771195 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:54471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:09:21.309660 7862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:09:21.309985 7862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:09:21.310196 7862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:09:21.320452 7862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:09:21.320713 7862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:09:21.320966 7862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:09:21.333841 7862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.63.84:54471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:45119
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:09:21.335618 7862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:09:21.337636 7862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:09:21.357604 7870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:21.357452 7871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:09:21.357403 7877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:09:21.359294 7862 server_base.cc:1034] running on GCE node
I20260806 01:09:21.360069 7862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:09:21.361287 7862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:09:21.362558 7862 hybrid_clock.cc:648] HybridClock initialized: now 1785978561362461 us; error 57 us; skew 500 ppm
I20260806 01:09:21.362919 7862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:09:21 dist-test-slave-7f01 krb5kdc[6476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978561, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:09:21.367888 7862 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:09:21.370119 7862 webserver.cc:460] Webserver started at http://127.22.63.67:38897/ using document root <none> and password file <none>
I20260806 01:09:21.371083 7862 fs_manager.cc:359] Metadata directory not provided
I20260806 01:09:21.371304 7862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:09:21.371817 7862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:09:21 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978561, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:09:21.389313 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "ef6165855fdc462998f9b050a032fe9d"
format_stamp: "Formatted at 2026-08-06 01:09:21 on dist-test-slave-7f01"
server_key: "ceeeae7e27be4d7bb0ec20f1213d5924"
server_key_iv: "4067e40167a45de4d879309b307bdda6"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:21.390455 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "ef6165855fdc462998f9b050a032fe9d"
format_stamp: "Formatted at 2026-08-06 01:09:21 on dist-test-slave-7f01"
server_key: "ceeeae7e27be4d7bb0ec20f1213d5924"
server_key_iv: "4067e40167a45de4d879309b307bdda6"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:21.397694 7862 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260806 01:09:21.417531 7887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:21.419157 7862 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:09:21.419413 7862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "ef6165855fdc462998f9b050a032fe9d"
format_stamp: "Formatted at 2026-08-06 01:09:21 on dist-test-slave-7f01"
server_key: "ceeeae7e27be4d7bb0ec20f1213d5924"
server_key_iv: "4067e40167a45de4d879309b307bdda6"
server_key_version: "kuduclusterkey@0"
I20260806 01:09:21.419690 7862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:09:21.452605 7862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:09:21.457970 7862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:09:21.458549 7862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:09:21.460064 7862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:09:21.462486 7862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:09:21.462574 7862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:21.462736 7862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:09:21.462859 7862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:09:21.532399 7862 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:43641
I20260806 01:09:21.532589 8069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:43641 every 8 connection(s)
I20260806 01:09:21.534042 7862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:09:21.540105 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 7862
I20260806 01:09:21.540343 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:09:21.540915 22781 external_mini_cluster.cc:1428] Setting key e4c484540d9467519ac60adb0b17730e
Aug 06 01:09:21 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978561, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:09:21.569162 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.538720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:50439 (local address 127.22.63.126:45119)
0806 01:09:21.540139 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.540148 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.541429 (+ 1281us) server_negotiation.cc:407] Connection header received
0806 01:09:21.546405 (+ 4976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.546410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.546481 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.546723 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:21.550584 (+ 3861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.551338 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.553760 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.554726 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.554879 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.560638 (+ 5759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:21.560645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:21.560648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:21.560686 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:21.564667 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:21.565652 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:21.565657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:21.565659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:21.565937 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:21.566526 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:21.566530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:21.566532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:21.567071 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:21.567193 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.568136 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.568393 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1182,"thread_start_us":735,"threads_started":1}
I20260806 01:09:21.570194 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.539691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45119 (local address 127.22.63.67:50439)
0806 01:09:21.541118 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.541161 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.546020 (+ 4859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.547209 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.547236 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.548683 (+ 1447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:09:21.550299 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.550330 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.551555 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.551559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.553542 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.553556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.555157 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.555166 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.555532 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.556779 (+ 1247us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:09:21.556811 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:09:21.560324 (+ 3513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:09:21.564829 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:21.564837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:21.564860 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:21.565531 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:21.566049 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:09:21.566052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:09:21.566055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:09:21.566379 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:09:21.567200 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:09:21.567207 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:09:21.567575 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.567915 (+ 340us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.568497 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":703,"threads_started":1}
I20260806 01:09:21.571720 8070 heartbeater.cc:347] Connected to a master server at 127.22.63.126:45119
I20260806 01:09:21.572184 8070 heartbeater.cc:464] Registering TS with master...
I20260806 01:09:21.573318 8070 heartbeater.cc:511] Master 127.22.63.126:45119 requested a full tablet report, sending...
I20260806 01:09:21.575083 7273 ts_manager.cc:194] Registered new tserver with Master: ef6165855fdc462998f9b050a032fe9d (127.22.63.67:43641)
I20260806 01:09:21.576085 7273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:50439
I20260806 01:09:21.583611 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:09:21.609258 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.591595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.22.63.126:45119)
0806 01:09:21.591804 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.591810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.592838 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 01:09:21.593007 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.593011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.593120 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.593205 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:21.595342 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.596343 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.598013 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.598695 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.598793 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.602112 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:21.602117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:21.602120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:21.602153 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:21.606026 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:21.606911 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:21.606915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:21.606917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:21.607141 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:21.607704 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:21.607708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:21.607710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:21.608120 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:21.608363 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.608713 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.608919 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:09:21.613994 7273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:09:21.626268 7273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:09:21.675462 8094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.644680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:41443 (local address 127.0.0.1:47188)
0806 01:09:21.658426 (+ 13746us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.658519 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.658986 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.661601 (+ 2615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.661609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.661694 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.663848 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.663864 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.667660 (+ 3796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.667665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.671487 (+ 3822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.671501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.673753 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.673762 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.674142 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.674234 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.674364 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.674659 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13330,"mutex_wait_us":27,"thread_start_us":12954,"threads_started":1}
I20260806 01:09:21.679018 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.644986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.22.63.67:43641)
0806 01:09:21.662696 (+ 17710us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.662709 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.663993 (+ 1284us) server_negotiation.cc:407] Connection header received
0806 01:09:21.664185 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.664192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.664439 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.665068 (+ 629us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.667414 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.669108 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.674270 (+ 5162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.677039 (+ 2769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.677186 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.677533 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.677961 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.678227 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17430,"thread_start_us":17068,"threads_started":1}
I20260806 01:09:21.679826 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.645639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.22.63.65:41235)
0806 01:09:21.663543 (+ 17904us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.663554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.663584 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:09:21.663772 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.663782 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.664099 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.664424 (+ 325us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.666248 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.669159 (+ 2911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.674612 (+ 5453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.677017 (+ 2405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.677186 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.677533 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.678753 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.679041 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17596,"thread_start_us":17289,"threads_started":1}
I20260806 01:09:21.680136 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.645535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43641 (local address 127.0.0.1:44834)
0806 01:09:21.663844 (+ 18309us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.663859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.664033 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.665111 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.665116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.665157 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.667037 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.667066 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.669293 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.669298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.673964 (+ 4666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.674009 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.677349 (+ 3340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.677358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.677706 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.677757 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.677916 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.678101 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18120,"mutex_wait_us":25,"thread_start_us":14262,"threads_started":1}
I20260806 01:09:21.680244 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.644748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:41235 (local address 127.0.0.1:56092)
0806 01:09:21.660485 (+ 15737us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.660508 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.660671 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.664365 (+ 3694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.664370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.664393 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.665741 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.665756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.669409 (+ 3653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.669425 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.674344 (+ 4919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.674360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.677914 (+ 3554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.677922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.678208 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.678242 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.678341 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.678537 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15454,"mutex_wait_us":56,"thread_start_us":17796,"threads_started":1}
I20260806 01:09:21.684602 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.644496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.22.63.66:41443)
0806 01:09:21.660316 (+ 15820us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.660326 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.660356 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:09:21.660470 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.660482 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.660827 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.661108 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.664144 (+ 3036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.667416 (+ 3272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.671701 (+ 4285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.673333 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.673538 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.673867 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.683943 (+ 10076us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.684104 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15586,"thread_start_us":15671,"threads_started":1}
I20260806 01:09:21.685300 7963 tablet_service.cc:1467] Processing CreateTablet for tablet 605d5f2b802b4038abd9351653d0180d (DEFAULT_TABLE table=test-table [id=bc3e5f6877a14b72a930790d0eb5406b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:09:21.685714 7506 tablet_service.cc:1467] Processing CreateTablet for tablet 605d5f2b802b4038abd9351653d0180d (DEFAULT_TABLE table=test-table [id=bc3e5f6877a14b72a930790d0eb5406b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:09:21.687551 7506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605d5f2b802b4038abd9351653d0180d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:09:21.687563 7963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605d5f2b802b4038abd9351653d0180d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:09:21.689219 7744 tablet_service.cc:1467] Processing CreateTablet for tablet 605d5f2b802b4038abd9351653d0180d (DEFAULT_TABLE table=test-table [id=bc3e5f6877a14b72a930790d0eb5406b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:09:21.691107 7744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 605d5f2b802b4038abd9351653d0180d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:09:21.704807 8104 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Bootstrap starting.
I20260806 01:09:21.710099 8104 tablet_bootstrap.cc:654] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Neither blocks nor log segments found. Creating new log.
I20260806 01:09:21.712597 8106 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Bootstrap starting.
I20260806 01:09:21.715619 8107 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Bootstrap starting.
I20260806 01:09:21.716745 8104 log.cc:824] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:09:21.718874 8106 tablet_bootstrap.cc:654] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Neither blocks nor log segments found. Creating new log.
I20260806 01:09:21.721212 8106 log.cc:824] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:09:21.721457 8107 tablet_bootstrap.cc:654] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Neither blocks nor log segments found. Creating new log.
I20260806 01:09:21.722465 8104 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: No bootstrap required, opened a new log
I20260806 01:09:21.722954 8104 ts_tablet_manager.cc:1397] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.006s
I20260806 01:09:21.724172 8107 log.cc:824] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Log is configured to *not* fsync() on all Append() calls
I20260806 01:09:21.726541 8106 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: No bootstrap required, opened a new log
I20260806 01:09:21.726943 8106 ts_tablet_manager.cc:1397] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260806 01:09:21.729482 8107 tablet_bootstrap.cc:492] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: No bootstrap required, opened a new log
I20260806 01:09:21.729923 8107 ts_tablet_manager.cc:1397] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
I20260806 01:09:21.736711 8104 raft_consensus.cc:348] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.737294 8104 raft_consensus.cc:374] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:09:21.737432 8104 raft_consensus.cc:729] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a049e8e86840faa3a27b6d389f03c8, State: Initialized, Role: FOLLOWER
I20260806 01:09:21.738134 8104 consensus_queue.cc:260] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.741709 8104 ts_tablet_manager.cc:1428] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Time spent starting tablet: real 0.019s user 0.010s sys 0.010s
I20260806 01:09:21.745522 8106 raft_consensus.cc:348] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.746316 8106 raft_consensus.cc:374] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:09:21.746554 8106 raft_consensus.cc:729] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef6165855fdc462998f9b050a032fe9d, State: Initialized, Role: FOLLOWER
I20260806 01:09:21.747443 8106 consensus_queue.cc:260] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.747975 7854 heartbeater.cc:503] Master 127.22.63.126:45119 was elected leader, sending a full tablet report...
I20260806 01:09:21.747818 8107 raft_consensus.cc:348] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.748463 8107 raft_consensus.cc:374] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:09:21.748668 8107 raft_consensus.cc:729] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f5ed91cf9b4184a14c1965bc48a710, State: Initialized, Role: FOLLOWER
I20260806 01:09:21.749534 8107 consensus_queue.cc:260] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.749917 8070 heartbeater.cc:503] Master 127.22.63.126:45119 was elected leader, sending a full tablet report...
I20260806 01:09:21.751001 8106 ts_tablet_manager.cc:1428] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Time spent starting tablet: real 0.024s user 0.014s sys 0.011s
I20260806 01:09:21.752205 8107 ts_tablet_manager.cc:1428] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Time spent starting tablet: real 0.022s user 0.020s sys 0.004s
W20260806 01:09:21.787214 8071 tablet.cc:2369] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:09:21.928761 7625 tablet.cc:2369] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:09:21.934284 8120 raft_consensus.cc:482] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:09:21.934719 8120 raft_consensus.cc:504] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.936872 8119 raft_consensus.cc:482] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:09:21.937139 8120 leader_election.cc:288] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443), 81a049e8e86840faa3a27b6d389f03c8 (127.22.63.65:41235)
I20260806 01:09:21.937453 8119 raft_consensus.cc:504] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.947768 8119 leader_election.cc:288] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443), ef6165855fdc462998f9b050a032fe9d (127.22.63.67:43641)
W20260806 01:09:21.949144 7855 tablet.cc:2369] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:09:21.952562 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.938797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:34929 (local address 127.22.63.65:41235)
0806 01:09:21.939037 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.939043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.939064 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:09:21.939152 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.939163 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.939250 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.939331 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.940092 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.941513 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.945852 (+ 4339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.948326 (+ 2474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.948561 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.948665 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.951949 (+ 3284us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.952156 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 01:09:21.953332 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.937492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:41235 (local address 127.22.63.67:34929)
0806 01:09:21.937715 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.937739 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.937893 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.939381 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.939386 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.939416 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.939935 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.939947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.941804 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.941810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.945618 (+ 3808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.945641 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.950853 (+ 5212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.950860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.951105 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.951157 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.951380 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.951689 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":3}
I20260806 01:09:21.955287 7546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "ef6165855fdc462998f9b050a032fe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a049e8e86840faa3a27b6d389f03c8" is_pre_election: true
I20260806 01:09:21.958217 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.938459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:39669 (local address 127.22.63.66:41443)
0806 01:09:21.938813 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.938819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.939063 (+ 244us) server_negotiation.cc:407] Connection header received
0806 01:09:21.939274 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.939277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.939370 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.939468 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.941152 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.942865 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.954559 (+ 11694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.956786 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.957023 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.957102 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.957643 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.957790 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":105}
I20260806 01:09:21.958726 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.937562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:41443 (local address 127.22.63.67:39669)
0806 01:09:21.938904 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.938923 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.939132 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.939546 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.939565 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.939608 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.940884 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.940899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.948358 (+ 7459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.948367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.954290 (+ 5923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.954314 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.957175 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.957181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.957437 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.957490 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.957610 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.957808 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":127,"thread_start_us":1009,"threads_started":1}
I20260806 01:09:21.957374 7546 raft_consensus.cc:2454] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef6165855fdc462998f9b050a032fe9d in term 0.
I20260806 01:09:21.960119 7894 leader_election.cc:302] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a049e8e86840faa3a27b6d389f03c8, ef6165855fdc462998f9b050a032fe9d; no voters:
I20260806 01:09:21.960302 7778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "ef6165855fdc462998f9b050a032fe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f5ed91cf9b4184a14c1965bc48a710" is_pre_election: true
I20260806 01:09:21.961443 7778 raft_consensus.cc:2454] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef6165855fdc462998f9b050a032fe9d in term 0.
I20260806 01:09:21.962316 8120 raft_consensus.cc:2781] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:09:21.962612 8120 raft_consensus.cc:482] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:09:21.962831 8120 raft_consensus.cc:3037] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:09:21.966265 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.942779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:42927 (local address 127.22.63.66:41443)
0806 01:09:21.950784 (+ 8005us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.950792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.950849 (+ 57us) server_negotiation.cc:407] Connection header received
0806 01:09:21.950912 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.950916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.951001 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.951109 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.953383 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.955959 (+ 2576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.962994 (+ 7035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.964553 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.964863 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.964970 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.965436 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.965616 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":7801,"thread_start_us":8068,"threads_started":1}
I20260806 01:09:21.966312 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.942088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:41443 (local address 127.22.63.65:42927)
0806 01:09:21.949058 (+ 6970us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.949083 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.949268 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.951979 (+ 2711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.951984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.952014 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.953105 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.953120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.956168 (+ 3048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.956174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.962726 (+ 6552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.962742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.965070 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.965078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.965255 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.965282 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.965395 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.965543 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6754,"thread_start_us":6781,"threads_started":1}
I20260806 01:09:21.967211 7778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "81a049e8e86840faa3a27b6d389f03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f5ed91cf9b4184a14c1965bc48a710" is_pre_election: true
I20260806 01:09:21.967499 7778 raft_consensus.cc:2454] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81a049e8e86840faa3a27b6d389f03c8 in term 0.
I20260806 01:09:21.968063 7442 leader_election.cc:302] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78f5ed91cf9b4184a14c1965bc48a710, 81a049e8e86840faa3a27b6d389f03c8; no voters:
I20260806 01:09:21.968652 8119 raft_consensus.cc:2781] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:09:21.968910 8119 raft_consensus.cc:482] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:09:21.969141 8119 raft_consensus.cc:3037] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:09:21.969072 8120 raft_consensus.cc:504] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.970709 7546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "ef6165855fdc462998f9b050a032fe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a049e8e86840faa3a27b6d389f03c8"
I20260806 01:09:21.970985 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.942229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:52495 (local address 127.22.63.67:43641)
0806 01:09:21.942566 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:21.942578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:21.955429 (+ 12851us) server_negotiation.cc:407] Connection header received
0806 01:09:21.955716 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:21.955722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:21.955845 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:21.955944 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:09:21.960397 (+ 4453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.962159 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.967428 (+ 5269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.969479 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.969684 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.969754 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:21.970512 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 01:09:21.970655 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:09:21.970803 7778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "ef6165855fdc462998f9b050a032fe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f5ed91cf9b4184a14c1965bc48a710"
I20260806 01:09:21.971491 7778 raft_consensus.cc:3037] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:09:21.971572 8120 leader_election.cc:288] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [CANDIDATE]: Term 1 election: Requested vote from peers 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443), 81a049e8e86840faa3a27b6d389f03c8 (127.22.63.65:41235)
I20260806 01:09:21.972121 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:21.943228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43641 (local address 127.22.63.65:52495)
0806 01:09:21.955299 (+ 12071us) negotiation.cc:107] Waiting for socket to connect
0806 01:09:21.955348 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 01:09:21.955568 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:09:21.956208 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:09:21.956212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:09:21.956266 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:09:21.960136 (+ 3870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.960151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.962527 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.962545 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.967122 (+ 4577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:09:21.967139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:09:21.970090 (+ 2951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:09:21.970098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:09:21.970299 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:21.970335 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 01:09:21.971148 (+ 813us) client_negotiation.cc:240] Negotiation successful
0806 01:09:21.971294 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11827,"thread_start_us":13282,"threads_started":1}
I20260806 01:09:21.974912 8119 raft_consensus.cc:504] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.975610 7546 raft_consensus.cc:2379] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ef6165855fdc462998f9b050a032fe9d in current term 1: Already voted for candidate 81a049e8e86840faa3a27b6d389f03c8 in this term.
I20260806 01:09:21.977130 8119 leader_election.cc:288] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [CANDIDATE]: Term 1 election: Requested vote from peers 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443), ef6165855fdc462998f9b050a032fe9d (127.22.63.67:43641)
I20260806 01:09:21.977222 7777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "81a049e8e86840faa3a27b6d389f03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f5ed91cf9b4184a14c1965bc48a710"
I20260806 01:09:21.977160 7994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "81a049e8e86840faa3a27b6d389f03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef6165855fdc462998f9b050a032fe9d" is_pre_election: true
I20260806 01:09:21.978190 7994 raft_consensus.cc:2379] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 81a049e8e86840faa3a27b6d389f03c8 in current term 1: Already voted for candidate ef6165855fdc462998f9b050a032fe9d in this term.
I20260806 01:09:21.978423 7993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "605d5f2b802b4038abd9351653d0180d" candidate_uuid: "81a049e8e86840faa3a27b6d389f03c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef6165855fdc462998f9b050a032fe9d"
I20260806 01:09:21.979352 7993 raft_consensus.cc:2379] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 81a049e8e86840faa3a27b6d389f03c8 in current term 1: Already voted for candidate ef6165855fdc462998f9b050a032fe9d in this term.
I20260806 01:09:21.978619 7778 raft_consensus.cc:2454] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef6165855fdc462998f9b050a032fe9d in term 1.
I20260806 01:09:21.980062 7889 leader_election.cc:302] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78f5ed91cf9b4184a14c1965bc48a710, ef6165855fdc462998f9b050a032fe9d; no voters:
I20260806 01:09:21.980629 8120 raft_consensus.cc:2781] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:09:21.980651 7445 leader_election.cc:302] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 81a049e8e86840faa3a27b6d389f03c8; no voters: 78f5ed91cf9b4184a14c1965bc48a710, ef6165855fdc462998f9b050a032fe9d
I20260806 01:09:21.981102 8119 raft_consensus.cc:2726] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 01:09:21.981772 8120 raft_consensus.cc:686] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 LEADER]: Becoming Leader. State: Replica: ef6165855fdc462998f9b050a032fe9d, State: Running, Role: LEADER
I20260806 01:09:21.982499 8120 consensus_queue.cc:237] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } }
I20260806 01:09:21.988556 7269 catalog_manager.cc:5495] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d reported cstate change: term changed from 0 to 1, leader changed from <none> to ef6165855fdc462998f9b050a032fe9d (127.22.63.67). New cstate: current_term: 1 leader_uuid: "ef6165855fdc462998f9b050a032fe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef6165855fdc462998f9b050a032fe9d" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 } health_report { overall_health: UNKNOWN } } }
I20260806 01:09:22.045938 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:22.037853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.22.63.67:43641)
0806 01:09:22.038053 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:22.038058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:22.038077 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:09:22.038216 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:22.038221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:22.038315 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:22.038622 (+ 307us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:09:22.039341 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.040679 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.042890 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.043563 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.043730 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:22.044394 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:09:22.045021 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:09:22.045184 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:22.045244 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 01:09:22.045341 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:09:22.056510 7778 raft_consensus.cc:1261] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 1 FOLLOWER]: Refusing update from remote peer ef6165855fdc462998f9b050a032fe9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:09:22.056664 7546 raft_consensus.cc:1261] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Refusing update from remote peer ef6165855fdc462998f9b050a032fe9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:09:22.057646 8131 consensus_queue.cc:1035] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "78f5ed91cf9b4184a14c1965bc48a710" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:09:22.058308 8120 consensus_queue.cc:1035] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a049e8e86840faa3a27b6d389f03c8" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:09:22.075601 8136 mvcc.cc:205] Tried to move back new op lower bound from 7315368190170402816 to 7315368189885005824. Current Snapshot: MvccSnapshot[applied={T|T < 7315368190170402816}]
I20260806 01:09:22.094281 7269 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:09:22.094842 7269 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:09:22.100375 7269 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b2869a152524d6fb511eea9f7a5ff3a: Sending DeleteTablet for 3 replicas of tablet 605d5f2b802b4038abd9351653d0180d
I20260806 01:09:22.102108 7963 tablet_service.cc:1514] Processing DeleteTablet for tablet 605d5f2b802b4038abd9351653d0180d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:09:22 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:44834
I20260806 01:09:22.102005 7744 tablet_service.cc:1514] Processing DeleteTablet for tablet 605d5f2b802b4038abd9351653d0180d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:09:22 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:47188
I20260806 01:09:22.102200 7506 tablet_service.cc:1514] Processing DeleteTablet for tablet 605d5f2b802b4038abd9351653d0180d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:09:22 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:56092
I20260806 01:09:22.109046 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 7405
I20260806 01:09:22.115106 8150 tablet_replica.cc:330] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: stopping tablet replica
I20260806 01:09:22.116148 8151 tablet_replica.cc:330] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: stopping tablet replica
I20260806 01:09:22.116823 8153 tablet_replica.cc:330] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: stopping tablet replica
I20260806 01:09:22.117295 8150 raft_consensus.cc:2229] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:09:22.117344 8153 raft_consensus.cc:2229] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:09:22.117563 8151 raft_consensus.cc:2229] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:09:22.118361 8153 raft_consensus.cc:2258] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:09:22.119019 8150 raft_consensus.cc:2258] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:09:22.121223 8153 ts_tablet_manager.cc:1905] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:09:22.122094 8150 ts_tablet_manager.cc:1905] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:09:22.119659 8151 raft_consensus.cc:2258] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:09:22.126701 8151 ts_tablet_manager.cc:1905] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:09:22.129460 8153 ts_tablet_manager.cc:1918] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:09:22.129707 8153 log.cc:1196] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/605d5f2b802b4038abd9351653d0180d
I20260806 01:09:22.130329 8153 ts_tablet_manager.cc:1939] T 605d5f2b802b4038abd9351653d0180d P 81a049e8e86840faa3a27b6d389f03c8: Deleting consensus metadata
I20260806 01:09:22.131500 8150 ts_tablet_manager.cc:1918] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:09:22.131860 8150 log.cc:1196] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/605d5f2b802b4038abd9351653d0180d
I20260806 01:09:22.132627 7249 catalog_manager.cc:4841] TS 81a049e8e86840faa3a27b6d389f03c8 (127.22.63.65:41235): tablet 605d5f2b802b4038abd9351653d0180d (table test-table [id=bc3e5f6877a14b72a930790d0eb5406b]) successfully deleted
I20260806 01:09:22.132805 8150 ts_tablet_manager.cc:1939] T 605d5f2b802b4038abd9351653d0180d P 78f5ed91cf9b4184a14c1965bc48a710: Deleting consensus metadata
I20260806 01:09:22.135210 7240 catalog_manager.cc:4841] TS 78f5ed91cf9b4184a14c1965bc48a710 (127.22.63.66:41443): tablet 605d5f2b802b4038abd9351653d0180d (table test-table [id=bc3e5f6877a14b72a930790d0eb5406b]) successfully deleted
Aug 06 01:09:22 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:09:22.137697 8151 ts_tablet_manager.cc:1918] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:09:22.138235 8151 log.cc:1196] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/605d5f2b802b4038abd9351653d0180d
I20260806 01:09:22.139004 8151 ts_tablet_manager.cc:1939] T 605d5f2b802b4038abd9351653d0180d P ef6165855fdc462998f9b050a032fe9d: Deleting consensus metadata
I20260806 01:09:22.140875 7247 catalog_manager.cc:4841] TS ef6165855fdc462998f9b050a032fe9d (127.22.63.67:43641): tablet 605d5f2b802b4038abd9351653d0180d (table test-table [id=bc3e5f6877a14b72a930790d0eb5406b]) successfully deleted
I20260806 01:09:22.148896 8101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:22.111016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.22.63.65:41235)
0806 01:09:22.111207 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:22.111213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:22.119562 (+ 8349us) server_negotiation.cc:407] Connection header received
0806 01:09:22.120018 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:22.120024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:22.120123 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:22.120255 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:22.123189 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.124876 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.127893 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.128957 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.129076 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:22.137944 (+ 8868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:22.137962 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:22.137981 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:22.138029 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:22.144308 (+ 6279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.145263 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.145274 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.145279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.145505 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.146006 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.146012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.146015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.146776 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:22.147378 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:22.147671 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 01:09:22.148128 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:09:22.149633 7615 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:09:22.247690 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 7405
I20260806 01:09:22.282130 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 7635
Aug 06 01:09:22 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:09:22.298137 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:22.283012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.22.63.66:41443)
0806 01:09:22.283274 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:22.283280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:22.283301 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:09:22.283463 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:22.283467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:22.283535 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:22.283655 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:22.285033 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.285754 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.287020 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.287447 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.287590 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:22.292038 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:22.292051 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:22.292066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:22.292104 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:22.295198 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.295988 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.295996 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.295998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.296142 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.296546 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.296550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.296552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.296911 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:22.297225 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:22.297561 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 01:09:22.297804 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260806 01:09:22.298791 7846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:09:22.392974 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 7635
I20260806 01:09:22.417074 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 7862
Aug 06 01:09:22 dist-test-slave-7f01 krb5kdc[6476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:09:22.434150 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:09:22.417913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.22.63.67:43641)
0806 01:09:22.418125 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:09:22.418133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:09:22.418170 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:09:22.418393 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:09:22.418406 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:09:22.418483 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:09:22.418590 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:09:22.420042 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.420825 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.422155 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:09:22.422661 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:09:22.422825 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:09:22.427852 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:09:22.427865 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:09:22.427880 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:09:22.427924 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:09:22.431137 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.431891 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.431908 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.431911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.432068 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:09:22.432481 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:09:22.432486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:09:22.432488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:09:22.432897 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:09:22.433270 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0806 01:09:22.433571 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 01:09:22.433810 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:09:22.434842 8061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:09:22.561084 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 7862
I20260806 01:09:22.587647 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 7214
I20260806 01:09:22.588951 7315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:09:22.706598 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 7214
I20260806 01:09:22.738173 22781 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:09:23.083808 22781 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:09:23.084127 22781 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:09:23.430223 22781 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:09:23.430 UTC [6529] LOG: received smart shutdown request
2026-08-06 01:09:23.434 UTC [6529] LOG: background worker "logical replication launcher" (PID 6545) exited with exit code 1
2026-08-06 01:09:23.434 UTC [6534] LOG: shutting down
2026-08-06 01:09:23.608 UTC [6529] LOG: database system is shut down
2026-08-06T01:09:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52142 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:09:23 dist-test-slave-7f01 krb5kdc[8174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:09:23 dist-test-slave-7f01 krb5kdc[8174](info): set up 2 sockets
Aug 06 01:09:23 dist-test-slave-7f01 krb5kdc[8174](info): commencing operation
krb5kdc: starting...
W20260806 01:09:25.096462 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.007s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:09:25 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:09:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangeradmin_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangeradmin_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangerlookup_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangerlookup_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
I20260806 01:09:25.516763 22781 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfkFK3E/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/postgres -l logfile start
2026-08-06 01:09:28.134 UTC [8217] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 01:09:28.134 UTC [8217] LOG: listening on IPv4 address "127.22.63.84", port 32843
2026-08-06 01:09:28.138 UTC [8217] LOG: listening on Unix socket "/tmp/.s.PGSQL.32843"
2026-08-06 01:09:28.162 UTC [8223] LOG: database system was shut down at 2026-08-06 01:09:26 UTC
2026-08-06 01:09:28.166 UTC [8217] LOG: database system is ready to accept connections
I20260806 01:09:29.339725 22781 mini_postgres.cc:96] Postgres bound to 32843
2026-08-06 01:09:29.364 UTC [8237] FATAL: database "slave" does not exist
127.22.63.84:32843 - accepting connections
I20260806 01:09:29.365165 22781 mini_ranger.cc:161] Starting Ranger...
I20260806 01:09:29.396620 22781 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 01:09:29.934763 22781 mini_ranger.cc:87] Created ranger Postgres database
I20260806 01:09:29.934926 22781 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin
2026-08-06 01:09:30,229 [I] DB FLAVOR :POSTGRES
2026-08-06 01:09:30,230 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:09:30,230 [I] Checking connection..
2026-08-06 01:09:30,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 01:09:30,470 [I] Checking connection passed.
2026-08-06 01:09:30,471 [I] --------- Verifying version history table ---------
2026-08-06 01:09:30,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:09:30,721 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:09:30,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:09:30,991 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 01:09:30,991 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 01:09:30,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 01:09:31.263 UTC [8355] WARNING: there is no transaction in progress
2026-08-06 01:09:31,272 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 01:09:31,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 01:09:31,526 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 01:09:31,526 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:09:31,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 01:09:31,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 01:09:32,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-06 01:09:32,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:09:32,548 [I] Table x_portal_user does not exist in database ranger
2026-08-06 01:09:32,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:09:32,805 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 01:09:32,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:09:33,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:09:33,355 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 01:09:33,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 01:09:34.334 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.351 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.368 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.384 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.403 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.424 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.490 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.497 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.512 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.523 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.533 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.549 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.559 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.566 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.573 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:34.581 UTC [8539] WARNING: there is no transaction in progress
2026-08-06 01:09:35,255 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 01:09:35,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 01:09:35,515 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 01:09:35,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 01:09:35,777 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 01:09:35,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 01:09:36,040 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 01:09:36,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 01:09:36,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 01:09:36,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-06 01:09:36,805 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 01:09:36,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 01:09:37,076 [I] DB_PATCHES have already been applied
I20260806 01:09:37.082890 22781 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfkFK3E/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 01:09:37.082983 22781 mini_ranger.cc:193] Using host: 127.22.63.84
I20260806 01:09:37.103873 22781 mini_ranger.cc:227] Ranger admin URL: http://127.22.63.84:54701
Aug 06, 2026 1:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 1:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 1:09:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 1:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 1:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 01:09:38.510632 22781 mini_ranger.cc:160] Time spent starting Ranger: real 9.145s user 0.011s sys 0.050s
Aug 06, 2026 1:09:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 1:09:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 1:09:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 1:09:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 1:09:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 01:09:59.851498 22781 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.63.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.63.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangerkms_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.63.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/rangerkms_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.63.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.63.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.63.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/HTTP_127.22.63.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 01:10:00 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978600, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 01:10:00.085196 22781 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 01:10:00.117805 22781 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 01:10:01.074074 22781 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 01:10:01.074177 22781 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms
I20260806 01:10:01.074218 22781 mini_ranger_kms.cc:227] Using postgres at 127.22.63.84:32843
2026-08-06 01:10:01,444 [I] DB FLAVOR :POSTGRES
2026-08-06 01:10:01,444 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 01:10:01,444 [I] Checking connection
2026-08-06 01:10:01,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 01:10:01,684 [I] connection success
2026-08-06 01:10:01,684 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 01:10:01,684 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 01:10:01,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 01:10:01,953 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 01:10:01,953 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 01:10:01,953 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 01:10:01,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.63.84:32843/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 01:10:02,254 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 01:10:02.829144 22781 mini_ranger_kms.cc:310] Created kms service
I20260806 01:10:03.040676 22781 mini_ranger_kms.cc:326] Created kudu user
I20260806 01:10:03.094365 22781 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 01:10:03 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978600, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:03.570573 22781 mini_ranger_kms.cc:384] Added ranger policy
I20260806 01:10:03.570938 22781 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfkFK3E/build/asan/bin/postgresql.jar:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfkFK3E/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 01:10:03.570986 22781 mini_ranger_kms.cc:240] Using host: 127.22.63.84
I20260806 01:10:03.597229 22781 mini_ranger_kms.cc:276] Ranger KMS PID: 8851
I20260806 01:10:03.597378 22781 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.63.84:48125
Aug 06, 2026 1:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 1:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 1:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 1:10:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 1:10:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 1:10:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 01:10:06 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978606, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:06.485810 22781 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.401s user 0.051s sys 0.367s
I20260806 01:10:06.486096 22781 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 01:10:06.486284 22781 mini_ranger_kms.cc:401] 127.22.63.84:48125/kms/v1/keys
Aug 06 01:10:06 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978606, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Aug 06 01:10:07 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978606, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.63.84@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Aug 06 01:10:09 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:10:09.463718 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41999
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:41999
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:09.819990 8932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:09.820294 8932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:09.820451 8932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:09.827858 8932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:09.828001 8932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:09.828079 8932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:09.828162 8932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:09.828254 8932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:09.838558 8932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.63.84:48125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:41999
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:09.840224 8932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:09.841714 8932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:09.856292 8941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:09.857009 8943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:09.857120 8932 server_base.cc:1034] running on GCE node
W20260806 01:10:09.857007 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:09.858314 8932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:09.860278 8932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:09.861552 8932 hybrid_clock.cc:648] HybridClock initialized: now 1785978609861481 us; error 56 us; skew 500 ppm
I20260806 01:10:09.861986 8932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:09 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:09.869505 8932 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:09.871258 8932 webserver.cc:460] Webserver started at http://127.22.63.126:33817/ using document root <none> and password file <none>
I20260806 01:10:09.872061 8932 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:09.872208 8932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:09.872534 8932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:10:09 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:09.940722 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf"
format_stamp: "Formatted at 2026-08-06 01:10:09 on dist-test-slave-7f01"
server_key: "b5f15272e5fd7ce6cc8d5709b4fc0beb"
server_key_iv: "ba79631e750f40f8d075f3becb7e2cdd"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:09.941690 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf"
format_stamp: "Formatted at 2026-08-06 01:10:09 on dist-test-slave-7f01"
server_key: "b5f15272e5fd7ce6cc8d5709b4fc0beb"
server_key_iv: "ba79631e750f40f8d075f3becb7e2cdd"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:09.947501 8932 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:10:09.969388 8963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:09.971364 8932 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:10:09.971601 8932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf"
format_stamp: "Formatted at 2026-08-06 01:10:09 on dist-test-slave-7f01"
server_key: "b5f15272e5fd7ce6cc8d5709b4fc0beb"
server_key_iv: "ba79631e750f40f8d075f3becb7e2cdd"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:09.971946 8932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:09.996779 8932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:10.001309 8932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:10.001689 8932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:10.022797 8932 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:41999
I20260806 01:10:10.022797 9050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:41999 every 8 connection(s)
I20260806 01:10:10.024340 8932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:10.027364 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 8932
I20260806 01:10:10.027604 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:10:10.028517 22781 external_mini_cluster.cc:1428] Setting key 9fdb7858cfd756cce6a77d239ed621c1
I20260806 01:10:10.029386 9052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:10.045156 9052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Bootstrap starting.
Aug 06 01:10:10 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:10.049065 9052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:10.050329 9052 log.cc:824] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:10.053747 9052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: No bootstrap required, opened a new log
I20260806 01:10:10.058321 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:10.030790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.22.63.126:41999)
0806 01:10:10.031620 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:10.031637 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:10.031693 (+ 56us) server_negotiation.cc:407] Connection header received
0806 01:10:10.034093 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:10.034125 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:10.034705 (+ 580us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:10.035214 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:10.037130 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:10.039227 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:10.041194 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:10.042436 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:10.042650 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:10.048124 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:10.048136 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:10.048159 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:10.048212 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:10.052643 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:10.053654 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:10.053664 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:10.053672 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:10.053878 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:10.054363 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:10.054368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:10.054372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:10.055152 (+ 780us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:10.055541 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:10.056099 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 01:10:10.056779 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":334,"threads_started":1}
I20260806 01:10:10.062644 9052 raft_consensus.cc:348] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:10.063047 9052 raft_consensus.cc:374] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:10.063163 9052 raft_consensus.cc:729] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17c3e8d4fc64441b8c8d2d26591d9bf, State: Initialized, Role: FOLLOWER
I20260806 01:10:10.063884 9052 consensus_queue.cc:260] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:10.064133 9052 raft_consensus.cc:388] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:10.064293 9052 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:10.064489 9052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:10.067101 9052 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:10.067715 9052 leader_election.cc:302] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a17c3e8d4fc64441b8c8d2d26591d9bf; no voters:
I20260806 01:10:10.068508 9052 leader_election.cc:288] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:10:10.068674 9061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:10.073755 9052 sys_catalog.cc:564] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:10.076459 9061 raft_consensus.cc:686] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 LEADER]: Becoming Leader. State: Replica: a17c3e8d4fc64441b8c8d2d26591d9bf, State: Running, Role: LEADER
I20260806 01:10:10.077105 9061 consensus_queue.cc:237] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:10.081321 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } } }
I20260806 01:10:10.081784 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:10.087637 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17c3e8d4fc64441b8c8d2d26591d9bf. Latest consensus state: current_term: 1 leader_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } } }
I20260806 01:10:10.088011 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:10.095175 9075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:10.102070 9075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:10.115011 9075 catalog_manager.cc:1347] Generated new cluster ID: 75faf023d06a4f99948aee0a915e99c8
I20260806 01:10:10.115147 9075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:10.127971 9052 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 01:10:10.143375 9075 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:10:10.147185 9075 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:10.164214 9075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Generated new TSK 0
I20260806 01:10:10.165454 9075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:10:11.028288 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:11 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 01:10:11.464329 9134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:11.464732 9134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:11.464886 9134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:11.476783 9134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:11.476930 9134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:11.477128 9134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:11.493137 9134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:11.494963 9134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:11.496801 9134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:11.514335 9149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:11.515184 9150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:11.523711 9154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:11.524264 9134 server_base.cc:1034] running on GCE node
I20260806 01:10:11.525537 9134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:11.528520 9134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:11.529953 9134 hybrid_clock.cc:648] HybridClock initialized: now 1785978611529831 us; error 99 us; skew 500 ppm
I20260806 01:10:11.530359 9134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:11 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:11.534911 9134 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:11.541710 9134 webserver.cc:460] Webserver started at http://127.22.63.65:34537/ using document root <none> and password file <none>
I20260806 01:10:11.542743 9134 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:11.542891 9134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:11.543258 9134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:10:11 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:11.565652 9134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "50ce867165ff43aa9a6881c85d570e28"
format_stamp: "Formatted at 2026-08-06 01:10:11 on dist-test-slave-7f01"
server_key: "26600b4123a35d32cdbd71e198beada2"
server_key_iv: "008962e41a810ade35ce9d53a96f1222"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:11.566890 9134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "50ce867165ff43aa9a6881c85d570e28"
format_stamp: "Formatted at 2026-08-06 01:10:11 on dist-test-slave-7f01"
server_key: "26600b4123a35d32cdbd71e198beada2"
server_key_iv: "008962e41a810ade35ce9d53a96f1222"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:11.574647 9134 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.009s
I20260806 01:10:11.604983 9167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:11.609292 9134 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.004s
I20260806 01:10:11.609579 9134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "50ce867165ff43aa9a6881c85d570e28"
format_stamp: "Formatted at 2026-08-06 01:10:11 on dist-test-slave-7f01"
server_key: "26600b4123a35d32cdbd71e198beada2"
server_key_iv: "008962e41a810ade35ce9d53a96f1222"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:11.609956 9134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:11.667302 9134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:11.673135 9134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:11.673655 9134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:11.675021 9134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:11.676930 9134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:11.677031 9134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:11.677270 9134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:11.677376 9134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:11.759158 9134 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:38997
I20260806 01:10:11.759199 9352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:38997 every 8 connection(s)
I20260806 01:10:11.760901 9134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:11.766264 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 9134
I20260806 01:10:11.766469 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:10:11.767333 22781 external_mini_cluster.cc:1428] Setting key 0c4a216b09897718e7975bcbb2948788
Aug 06 01:10:11 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:11.797281 9359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:11.764415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.65:45379)
0806 01:10:11.772386 (+ 7971us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:11.772463 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:11.774973 (+ 2510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:11.776366 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:11.776380 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:11.777055 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:11.777918 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:11.777943 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:11.780194 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:11.780199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:11.782184 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:11.782199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:11.783291 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:11.783298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:11.783591 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:11.784794 (+ 1203us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:11.784824 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:11.788229 (+ 3405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:11.792405 (+ 4176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:11.792415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:11.792434 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:11.793038 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:11.793513 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:11.793518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:11.793522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:11.793760 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:11.794507 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:11.794516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:11.794961 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 01:10:11.795271 (+ 310us) client_negotiation.cc:240] Negotiation successful
0806 01:10:11.795831 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7651,"thread_start_us":7411,"threads_started":1}
I20260806 01:10:11.798205 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:11.765091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:45379 (local address 127.22.63.126:41999)
0806 01:10:11.775758 (+ 10667us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:11.775765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:11.775785 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:10:11.775845 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:11.775850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:11.775946 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:11.778100 (+ 2154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:11.778264 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:11.779420 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:11.782384 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:11.783023 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:11.784061 (+ 1038us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:11.788524 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:11.788530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:11.788535 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:11.788580 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:11.792229 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:11.793190 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:11.793195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:11.793199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:11.793365 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:11.793895 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:11.793900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:11.793904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:11.794289 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:11.794420 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:11.797600 (+ 3180us) server_negotiation.cc:299] Negotiation successful
0806 01:10:11.797835 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10497,"thread_start_us":10375,"threads_started":1}
I20260806 01:10:11.800024 9357 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:11.800354 9357 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:11.801239 9357 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:11.803972 8990 ts_manager.cc:194] Registered new tserver with Master: 50ce867165ff43aa9a6881c85d570e28 (127.22.63.65:38997)
I20260806 01:10:11.809973 8990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:45379
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Aug 06 01:10:11 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:10:11.877066 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:12 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978611, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
W20260806 01:10:12.400628 9363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:12.401039 9363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:12.401247 9363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:12.413028 9363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:12.413242 9363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:12.413468 9363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:12.429826 9363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:12.431612 9363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:12.433432 9363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:12.449429 9385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:12.449455 9391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:12.449363 9387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:12.450280 9363 server_base.cc:1034] running on GCE node
I20260806 01:10:12.450917 9363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:12.451990 9363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:12.453219 9363 hybrid_clock.cc:648] HybridClock initialized: now 1785978612453177 us; error 35 us; skew 500 ppm
I20260806 01:10:12.453552 9363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:12 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:12.456905 9363 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:12.458362 9363 webserver.cc:460] Webserver started at http://127.22.63.66:34159/ using document root <none> and password file <none>
I20260806 01:10:12.459041 9363 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:12.459141 9363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:12.459404 9363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:10:12 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:12.475093 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "65cb671405094036aa280533ed0aaeab"
format_stamp: "Formatted at 2026-08-06 01:10:12 on dist-test-slave-7f01"
server_key: "f025eb09078e8adfd4dc6c105774f7b3"
server_key_iv: "621c755c3b081d120251007edc50bf5e"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:12.476263 9363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "65cb671405094036aa280533ed0aaeab"
format_stamp: "Formatted at 2026-08-06 01:10:12 on dist-test-slave-7f01"
server_key: "f025eb09078e8adfd4dc6c105774f7b3"
server_key_iv: "621c755c3b081d120251007edc50bf5e"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:12.483795 9363 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 01:10:12.500634 9403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:12.502413 9363 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:10:12.502632 9363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "65cb671405094036aa280533ed0aaeab"
format_stamp: "Formatted at 2026-08-06 01:10:12 on dist-test-slave-7f01"
server_key: "f025eb09078e8adfd4dc6c105774f7b3"
server_key_iv: "621c755c3b081d120251007edc50bf5e"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:12.503024 9363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:12.529410 9363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:12.534036 9363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:12.534368 9363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:12.535414 9363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:12.536969 9363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:12.537079 9363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:12.537230 9363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:12.537307 9363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:12.577484 9363 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:44407
I20260806 01:10:12.577466 9582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:44407 every 8 connection(s)
I20260806 01:10:12.579190 9363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:12.589483 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 9363
I20260806 01:10:12.589671 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:10:12.590245 22781 external_mini_cluster.cc:1428] Setting key da0fc1232da4a0f5fef6463a7d5edd99
Aug 06 01:10:12 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:12.602032 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:12.582242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:60919 (local address 127.22.63.126:41999)
0806 01:10:12.582790 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:12.582797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:12.583868 (+ 1071us) server_negotiation.cc:407] Connection header received
0806 01:10:12.586495 (+ 2627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:12.586503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:12.586604 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:12.586700 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:12.588759 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:12.589557 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:12.591431 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:12.591930 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:12.592069 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:12.595571 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:12.595576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:12.595580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:12.595613 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:12.598683 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:12.599558 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:12.599567 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:12.599570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:12.599699 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:12.600202 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:12.600211 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:12.600214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:12.600519 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:12.600678 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:12.601407 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 01:10:12.601637 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":263,"threads_started":1}
I20260806 01:10:12.603243 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:12.582762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.66:60919)
0806 01:10:12.583654 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:12.583712 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:12.586114 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:12.587001 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:12.587014 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:12.587641 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:12.588551 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:12.588577 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:12.589781 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:12.589787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:12.591284 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:12.591296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:12.592167 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:12.592175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:12.592477 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:12.593223 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:12.593252 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:12.595323 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:12.598865 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:12.598876 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:12.598894 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:12.599430 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:12.599858 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:12.599863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:12.599868 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:12.600082 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:12.600667 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:12.600675 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:12.601027 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 01:10:12.601319 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 01:10:12.601854 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":334,"threads_started":1}
I20260806 01:10:12.604831 9583 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:12.605155 9583 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:12.606001 9583 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:12.607919 8990 ts_manager.cc:194] Registered new tserver with Master: 65cb671405094036aa280533ed0aaeab (127.22.63.66:44407)
I20260806 01:10:12.609190 8990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:60919
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:10:12.661607 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:12.814184 9357 heartbeater.cc:503] Master 127.22.63.126:41999 was elected leader, sending a full tablet report...
W20260806 01:10:13.022293 9597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:13.022564 9597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:13.022724 9597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:13.030064 9597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:13.030171 9597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:13.030354 9597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:13.040143 9597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:13.041447 9597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:13.042743 9597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:13.055914 9614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:13.055925 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:13.056064 9607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:13.060599 9597 server_base.cc:1034] running on GCE node
I20260806 01:10:13.061259 9597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:13.062253 9597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:13.063454 9597 hybrid_clock.cc:648] HybridClock initialized: now 1785978613063386 us; error 59 us; skew 500 ppm
I20260806 01:10:13.063840 9597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:13.067102 9597 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:13.068678 9597 webserver.cc:460] Webserver started at http://127.22.63.67:41049/ using document root <none> and password file <none>
I20260806 01:10:13.069404 9597 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:13.069556 9597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:13.069890 9597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:13.086364 9597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "0fd25f8e463c4c919733f108a835626b"
format_stamp: "Formatted at 2026-08-06 01:10:13 on dist-test-slave-7f01"
server_key: "70a7052fe2c302ce40f1caa3a20722a8"
server_key_iv: "699a8b75c4f8c9850d880f79ef286b08"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:13.087608 9597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "0fd25f8e463c4c919733f108a835626b"
format_stamp: "Formatted at 2026-08-06 01:10:13 on dist-test-slave-7f01"
server_key: "70a7052fe2c302ce40f1caa3a20722a8"
server_key_iv: "699a8b75c4f8c9850d880f79ef286b08"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:13.095130 9597 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260806 01:10:13.111137 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:13.112737 9597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:10:13.112946 9597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "0fd25f8e463c4c919733f108a835626b"
format_stamp: "Formatted at 2026-08-06 01:10:13 on dist-test-slave-7f01"
server_key: "70a7052fe2c302ce40f1caa3a20722a8"
server_key_iv: "699a8b75c4f8c9850d880f79ef286b08"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:13.113234 9597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:13.156145 9597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:13.159947 9597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:13.160283 9597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:13.161262 9597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:13.162671 9597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:13.162787 9597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:13.162940 9597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:13.163020 9597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:13.198355 9597 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:45783
I20260806 01:10:13.198447 9808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:45783 every 8 connection(s)
I20260806 01:10:13.199661 9597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:13.205893 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 9597
I20260806 01:10:13.206125 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:10:13.206598 22781 external_mini_cluster.cc:1428] Setting key 5a8d2f05c8e928e46adbe089882d0882
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:13.220156 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:13.202458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:50961 (local address 127.22.63.126:41999)
0806 01:10:13.202979 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:13.202985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:13.203947 (+ 962us) server_negotiation.cc:407] Connection header received
0806 01:10:13.205868 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:13.205872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:13.205961 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:13.206099 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:13.207682 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.208593 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.210277 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.210716 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.210847 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:13.214074 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:13.214080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:13.214083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:13.214120 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:13.216960 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.217777 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.217781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.217783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.217915 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.218489 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.218497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.218500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.218811 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:13.218954 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:13.219545 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 01:10:13.219718 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":231,"threads_started":1}
I20260806 01:10:13.221261 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:13.202846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.67:50961)
0806 01:10:13.203709 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:13.203824 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:13.205547 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:13.206341 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:13.206348 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:13.206945 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:13.207509 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:13.207525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.208774 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.208778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:13.210128 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:13.210147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.210949 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.210956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:13.211186 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:13.211827 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:13.211848 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:13.213850 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:13.217122 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:13.217129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:13.217141 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:13.217660 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:13.218082 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:13.218094 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:13.218097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:13.218304 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:13.218927 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:13.218933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:13.219225 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 01:10:13.219461 (+ 236us) client_negotiation.cc:240] Negotiation successful
0806 01:10:13.219923 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":348,"threads_started":1}
I20260806 01:10:13.222782 9810 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:13.223101 9810 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:13.223886 9810 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:13.225286 8988 ts_manager.cc:194] Registered new tserver with Master: 0fd25f8e463c4c919733f108a835626b (127.22.63.67:45783)
I20260806 01:10:13.226255 8988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:50961
I20260806 01:10:13.234699 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:10:13.236469 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 9134
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:10:13.253177 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:13.237457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57734 (local address 127.22.63.65:38997)
0806 01:10:13.238100 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:13.238107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:13.238133 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:10:13.238197 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:13.238203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:13.238399 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:13.238540 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:13.240106 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.241126 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.242484 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.243020 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.243118 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:13.247419 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:13.247430 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:13.247443 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:13.247476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:13.250187 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.250939 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.250946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.250948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.251102 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.251493 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.251499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.251502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.251924 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:13.252235 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:13.252558 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 01:10:13.252816 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":328,"threads_started":1}
I20260806 01:10:13.254415 9345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:13.337630 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 9134
I20260806 01:10:13.359323 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 9363
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:10:13.375849 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:13.360329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.22.63.66:44407)
0806 01:10:13.360858 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:13.360865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:13.360890 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:10:13.360949 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:13.360956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:13.361174 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:13.361355 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:13.362737 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.363715 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.365198 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.365751 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.365889 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:13.370375 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:13.370386 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:13.370401 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:13.370436 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:13.373223 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.373861 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.373868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.373870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.373999 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.374379 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.374382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.374384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.374729 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:13.374979 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:13.375268 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 01:10:13.375481 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
I20260806 01:10:13.377163 9576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:13.461651 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 9363
I20260806 01:10:13.488600 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 9597
Aug 06 01:10:13 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:10:13.505385 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:13.489587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.22.63.67:45783)
0806 01:10:13.490261 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:13.490267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:13.490299 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:10:13.490362 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:13.490368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:13.490587 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:13.490723 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:13.492177 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.493238 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.494533 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:13.495091 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:13.495191 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:13.499942 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:13.499965 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:13.499982 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:13.500023 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:13.502758 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.503380 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.503387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.503389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.503523 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:13.504027 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:13.504031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:13.504033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:13.504385 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:13.504640 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:13.504876 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 01:10:13.505094 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":335,"threads_started":1}
I20260806 01:10:13.506737 9802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:13.591641 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 9597
I20260806 01:10:13.616496 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 8932
I20260806 01:10:13.617909 9044 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:13.729596 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 8932
I20260806 01:10:13.763167 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41999
--webserver_interface=127.22.63.126
--webserver_port=33817
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:41999
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:14.130035 9830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:14.130321 9830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:14.130496 9830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:14.138036 9830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:14.138151 9830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:14.138198 9830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:14.138258 9830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:14.138306 9830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:14.148654 9830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:41999
--ranger_config_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=33817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:14.149971 9830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:14.151396 9830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:14.164924 9838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:14.165095 9843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:14.165176 9841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:14.165995 9830 server_base.cc:1034] running on GCE node
I20260806 01:10:14.166824 9830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:14.168373 9830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:14.169735 9830 hybrid_clock.cc:648] HybridClock initialized: now 1785978614169629 us; error 69 us; skew 500 ppm
I20260806 01:10:14.170202 9830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:14 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:14.173748 9830 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:14.175287 9830 webserver.cc:460] Webserver started at http://127.22.63.126:33817/ using document root <none> and password file <none>
I20260806 01:10:14.176061 9830 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:14.176186 9830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:10:14 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:14.221330 9830 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:10:14.224709 9854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:14.226130 9830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:10:14.226315 9830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf"
format_stamp: "Formatted at 2026-08-06 01:10:09 on dist-test-slave-7f01"
server_key: "b5f15272e5fd7ce6cc8d5709b4fc0beb"
server_key_iv: "ba79631e750f40f8d075f3becb7e2cdd"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:14.227342 9830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:14.243824 9830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:14.247615 9830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:14.247963 9830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:14.267421 9830 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:41999
I20260806 01:10:14.267441 9927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:41999 every 8 connection(s)
I20260806 01:10:14.269090 9830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:14.273841 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 9830
I20260806 01:10:14.274945 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:38997
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:41999
--webserver_port=34537
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:14.278316 9928 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:10:14.283877 9928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Bootstrap starting.
I20260806 01:10:14.304553 9928 log.cc:824] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:14.314257 9928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:10:14.314832 9928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Bootstrap complete.
I20260806 01:10:14.323993 9928 raft_consensus.cc:348] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:14.324817 9928 raft_consensus.cc:729] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a17c3e8d4fc64441b8c8d2d26591d9bf, State: Initialized, Role: FOLLOWER
I20260806 01:10:14.325544 9928 consensus_queue.cc:260] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:14.325779 9928 raft_consensus.cc:388] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:14.325920 9928 raft_consensus.cc:482] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:14.326109 9928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:10:14.330179 9928 raft_consensus.cc:504] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:14.330737 9928 leader_election.cc:302] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a17c3e8d4fc64441b8c8d2d26591d9bf; no voters:
I20260806 01:10:14.331439 9928 leader_election.cc:288] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:10:14.331681 9939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:10:14.333009 9939 raft_consensus.cc:686] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [term 2 LEADER]: Becoming Leader. State: Replica: a17c3e8d4fc64441b8c8d2d26591d9bf, State: Running, Role: LEADER
I20260806 01:10:14.333647 9939 consensus_queue.cc:237] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } }
I20260806 01:10:14.333853 9928 sys_catalog.cc:564] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:14.336525 9944 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a17c3e8d4fc64441b8c8d2d26591d9bf. Latest consensus state: current_term: 2 leader_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } } }
I20260806 01:10:14.336930 9944 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:14.337441 9942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a17c3e8d4fc64441b8c8d2d26591d9bf" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41999 } } }
I20260806 01:10:14.337766 9942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:14.353282 9953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:14.360986 9953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:14.362443 9953 catalog_manager.cc:1259] Loaded cluster ID: 75faf023d06a4f99948aee0a915e99c8
I20260806 01:10:14.362619 9953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:14.366703 9953 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:14.368541 9953 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Loaded TSK: 0
I20260806 01:10:14.370217 9953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:10:14.374899 9928 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 01:10:14.737231 9933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:14.737725 9933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:14.737924 9933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:14.748998 9933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:14.749204 9933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:14.749428 9933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:14.764533 9933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:38997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=34537
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:14.766425 9933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:14.768294 9933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:14.795562 9999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:14.800190 10000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:14.807160 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:14.810449 9933 server_base.cc:1034] running on GCE node
I20260806 01:10:14.811352 9933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:14.814559 9933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:14.822587 9933 hybrid_clock.cc:648] HybridClock initialized: now 1785978614822488 us; error 80 us; skew 500 ppm
I20260806 01:10:14.823129 9933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:14 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:14.828058 9933 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:14.843901 9933 webserver.cc:460] Webserver started at http://127.22.63.65:34537/ using document root <none> and password file <none>
I20260806 01:10:14.845136 9933 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:14.845382 9933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:10:14 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:14.902109 9933 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.004s sys 0.012s
I20260806 01:10:14.906800 10015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:14.908882 9933 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 01:10:14.909163 9933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "50ce867165ff43aa9a6881c85d570e28"
format_stamp: "Formatted at 2026-08-06 01:10:11 on dist-test-slave-7f01"
server_key: "26600b4123a35d32cdbd71e198beada2"
server_key_iv: "008962e41a810ade35ce9d53a96f1222"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:14.910720 9933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:14.964283 9933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:14.969900 9933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:14.970265 9933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:14.971659 9933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:14.973650 9933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:14.973778 9933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:14.973942 9933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:14.974067 9933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:15.037212 9933 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:38997
I20260806 01:10:15.037226 10214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:38997 every 8 connection(s)
I20260806 01:10:15.039222 9933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:15.046291 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 9933
I20260806 01:10:15.047151 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:44407
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:41999
--webserver_port=34159
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:15 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978614, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:15.086681 10220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:15.055204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:37345 (local address 127.22.63.126:41999)
0806 01:10:15.058869 (+ 3665us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:15.058888 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:15.058940 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:10:15.059709 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:15.059727 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:15.060269 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:15.060681 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:15.063593 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.065661 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.068233 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.069332 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.069540 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:15.077858 (+ 8318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:15.077868 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:15.077885 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:15.077933 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:15.081485 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:15.082421 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:15.082429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:15.082434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:15.082607 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:15.083105 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:15.083109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:15.083111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:15.083777 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:15.084021 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:15.084831 (+ 810us) server_negotiation.cc:299] Negotiation successful
0806 01:10:15.085219 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3399,"thread_start_us":3186,"threads_started":1}
I20260806 01:10:15.087986 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:15.054146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.65:37345)
0806 01:10:15.055047 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:15.055104 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:15.059056 (+ 3952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:15.060841 (+ 1785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:15.060853 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:15.061520 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:15.062631 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:15.062659 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.065890 (+ 3231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.065896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:15.068004 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:15.068034 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.071726 (+ 3692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.071763 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:15.072090 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:15.074759 (+ 2669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:15.074795 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:15.077506 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:15.081668 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:15.081679 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:15.081695 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:15.082178 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:15.082741 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:15.082746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:15.082750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:15.082967 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:15.083935 (+ 968us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:15.083944 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:15.084297 (+ 353us) client_negotiation.cc:769] Sending connection context
0806 01:10:15.084587 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 01:10:15.085194 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":1199,"threads_started":1}
I20260806 01:10:15.092267 10216 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:15.092604 10216 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:15.093394 10216 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
W20260806 01:10:15.097080 10216 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:41999 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:10:15.097317 10216 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:15.097621 10216 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:15.099004 10216 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:15.099291 10216 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
W20260806 01:10:15.100493 10216 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:10:15.475872 10222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:15.476168 10222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:15.476325 10222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:15.484854 10222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:15.485074 10222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:15.485284 10222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:15.497710 10222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:44407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=34159
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:15.499143 10222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:15.500564 10222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:15.517560 10243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:15.518296 10244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:15.518330 10250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:15.519068 10222 server_base.cc:1034] running on GCE node
I20260806 01:10:15.519936 10222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:15.521099 10222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:15.522351 10222 hybrid_clock.cc:648] HybridClock initialized: now 1785978615522278 us; error 65 us; skew 500 ppm
I20260806 01:10:15.522768 10222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:15 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:15.526175 10222 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:15.528154 10222 webserver.cc:460] Webserver started at http://127.22.63.66:34159/ using document root <none> and password file <none>
I20260806 01:10:15.528889 10222 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:15.529063 10222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:10:15 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:15.549448 10222 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:10:15.553225 10265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:15.554811 10222 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:10:15.555014 10222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "65cb671405094036aa280533ed0aaeab"
format_stamp: "Formatted at 2026-08-06 01:10:12 on dist-test-slave-7f01"
server_key: "f025eb09078e8adfd4dc6c105774f7b3"
server_key_iv: "621c755c3b081d120251007edc50bf5e"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:15.556145 10222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:15.602864 10222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:15.608245 10222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:15.608625 10222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:15.611732 10222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:15.613587 10222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:15.613754 10222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:10:15.613910 10222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:15.614032 10222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:15.681190 10222 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:44407
I20260806 01:10:15.681263 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:44407 every 8 connection(s)
I20260806 01:10:15.683255 10222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:15.686004 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 10222
I20260806 01:10:15.686928 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.63.84:48125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:45783
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:41999
--webserver_port=41049
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:15 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 01:10:15 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:15.740116 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:15.700038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:39641 (local address 127.22.63.126:41999)
0806 01:10:15.716214 (+ 16176us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:15.716218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:15.716240 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:10:15.716302 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:15.716306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:15.716383 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:15.716546 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:15.718926 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.720201 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.722720 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.723423 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.723551 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:15.728806 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:15.728813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:15.728819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:15.728884 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:15.734504 (+ 5620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:15.735555 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:15.735560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:15.735563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:15.735715 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:15.736319 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:15.736324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:15.736328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:15.736699 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:15.736835 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:15.738458 (+ 1623us) server_negotiation.cc:299] Negotiation successful
0806 01:10:15.738690 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16042,"thread_start_us":15893,"threads_started":1}
I20260806 01:10:15.741911 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:15.700852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.66:39641)
0806 01:10:15.701924 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:15.701982 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:15.704787 (+ 2805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:15.716909 (+ 12122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:15.716924 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:15.717640 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:15.718594 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:15.718625 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.720428 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.720434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:15.722508 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:15.722525 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:15.723922 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:15.723933 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:15.724249 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:15.725096 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:15.725129 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:15.728490 (+ 3361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:15.734701 (+ 6211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:15.734712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:15.734731 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:15.735384 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:15.735893 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:15.735899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:15.735902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:15.736175 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:15.736935 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:15.736944 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:15.737382 (+ 438us) client_negotiation.cc:769] Sending connection context
0806 01:10:15.737698 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 01:10:15.739057 (+ 1359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":405,"threads_started":1}
I20260806 01:10:15.744180 10450 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:15.744506 10450 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:15.745424 10450 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:15.747786 9876 ts_manager.cc:194] Registered new tserver with Master: 65cb671405094036aa280533ed0aaeab (127.22.63.66:44407)
I20260806 01:10:15.749998 9876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:39641
I20260806 01:10:16.100677 10216 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:16.101176 10216 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:16.102629 9876 ts_manager.cc:194] Registered new tserver with Master: 50ce867165ff43aa9a6881c85d570e28 (127.22.63.65:38997)
I20260806 01:10:16.103940 9876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:37345
Aug 06 01:10:16 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
W20260806 01:10:16.154618 10454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:16.155073 10454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:16.155243 10454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:16.166560 10454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:16.166798 10454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:16.167032 10454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:16.182721 10454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.63.84:48125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:45783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=41049
--tserver_master_addrs=127.22.63.126:41999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:16.184777 10454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:16.186563 10454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:16.209519 10464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:16.212172 10463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:16.216315 10466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:16.217408 10454 server_base.cc:1034] running on GCE node
I20260806 01:10:16.218171 10454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:16.219393 10454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:16.220688 10454 hybrid_clock.cc:648] HybridClock initialized: now 1785978616220607 us; error 64 us; skew 500 ppm
I20260806 01:10:16.221076 10454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:16 dist-test-slave-7f01 krb5kdc[8174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:16.225246 10454 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:16.229218 10454 webserver.cc:460] Webserver started at http://127.22.63.67:41049/ using document root <none> and password file <none>
I20260806 01:10:16.230141 10454 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:16.230290 10454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 01:10:16 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:16.284579 10454 fs_manager.cc:709] Time spent opening directory manager: real 0.025s user 0.008s sys 0.008s
I20260806 01:10:16.300468 10475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:16.302573 10454 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.006s sys 0.010s
I20260806 01:10:16.302892 10454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "0fd25f8e463c4c919733f108a835626b"
format_stamp: "Formatted at 2026-08-06 01:10:13 on dist-test-slave-7f01"
server_key: "70a7052fe2c302ce40f1caa3a20722a8"
server_key_iv: "699a8b75c4f8c9850d880f79ef286b08"
server_key_version: "kuduclusterkey@0"
I20260806 01:10:16.304597 10454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:16.354913 10454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:16.359357 10454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:16.359683 10454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:16.361159 10454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:16.362922 10454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:16.363060 10454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:16.363183 10454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:16.363265 10454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:16.415386 10454 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:45783
I20260806 01:10:16.415444 10660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:45783 every 8 connection(s)
I20260806 01:10:16.417415 10454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:16.426874 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 10454
Aug 06 01:10:16 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:16.468204 10663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.428019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.22.63.126:41999)
0806 01:10:16.450123 (+ 22104us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.450127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.450142 (+ 15us) server_negotiation.cc:407] Connection header received
0806 01:10:16.450180 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.450184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.450250 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.450383 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:16.452949 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.453677 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.455844 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.456476 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.456599 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.460779 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:16.460787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:16.460796 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:16.460835 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:16.464917 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.465915 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.465921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.465924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.466112 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.466572 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.466577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.466580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.466896 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:16.467163 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.467565 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.467829 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21984,"thread_start_us":20807,"threads_started":1}
I20260806 01:10:16.471534 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.421269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:55957 (local address 127.22.63.126:41999)
0806 01:10:16.449124 (+ 27855us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.449133 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.449164 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:10:16.449250 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.449259 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.449391 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.449524 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:16.453909 (+ 4385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.455120 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.457604 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.458315 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.458423 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.462945 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:16.462954 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:16.462960 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:16.463024 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:16.466340 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.467259 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.467263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.467266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.467428 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.468089 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.468094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.468099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.468469 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:16.468593 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.469859 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.471183 (+ 1324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27675,"thread_start_us":28556,"threads_started":1}
I20260806 01:10:16.471578 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.421956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41999 (local address 127.22.63.67:55957)
0806 01:10:16.443925 (+ 21969us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.443992 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.447278 (+ 3286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.450973 (+ 3695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.450989 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.451687 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:16.452676 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.452702 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.455342 (+ 2640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.455347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.457434 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.457453 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.458859 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.458868 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.459231 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.460028 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:16.460087 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:16.462624 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:16.466500 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:16.466511 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:16.466531 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:16.467148 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:16.467570 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:16.467576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:16.467580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:16.467923 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:16.468683 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:16.468714 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:16.469238 (+ 524us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.469534 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.470140 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21513,"thread_start_us":25646,"threads_started":1}
I20260806 01:10:16.473191 10661 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41999
I20260806 01:10:16.473529 10661 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:16.474283 10661 heartbeater.cc:511] Master 127.22.63.126:41999 requested a full tablet report, sending...
I20260806 01:10:16.475898 9876 ts_manager.cc:194] Registered new tserver with Master: 0fd25f8e463c4c919733f108a835626b (127.22.63.67:45783)
I20260806 01:10:16.476683 9876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:55957
I20260806 01:10:16.483559 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 01:10:16 dist-test-slave-7f01 krb5kdc[8174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978615, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for HTTP/127.22.63.84@KRBTEST.COM
I20260806 01:10:16.530102 10665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.492956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.22.63.126:41999)
0806 01:10:16.493167 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.493172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.500385 (+ 7213us) server_negotiation.cc:407] Connection header received
0806 01:10:16.500958 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.500965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.501066 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.501150 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:16.503396 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.504544 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.506538 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.507275 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.507438 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.522548 (+ 15110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:16.522556 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:16.522562 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:16.522617 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:16.526425 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.527443 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.527449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.527452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.527611 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.528155 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.528160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.528164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.528560 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:16.528775 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.529482 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.529734 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:10:16.538197 9876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:10:16.550240 9876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:10:16.643322 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.622444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:44407 (local address 127.0.0.1:43246)
0806 01:10:16.628884 (+ 6440us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.628907 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.629048 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.630200 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.630203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.630245 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:16.630756 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.630780 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.633632 (+ 2852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.633637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.638577 (+ 4940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.638590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.642197 (+ 3607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.642206 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.642459 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.642529 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.642666 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.642879 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6304,"mutex_wait_us":1366,"spinlock_wait_cycles":6016,"thread_start_us":6674,"threads_started":1}
I20260806 01:10:16.645368 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.581078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.22.63.65:38997)
0806 01:10:16.611795 (+ 30717us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.611805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.611882 (+ 77us) server_negotiation.cc:407] Connection header received
0806 01:10:16.622983 (+ 11101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.622996 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.623263 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.623490 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:16.636253 (+ 12763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.637815 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.640272 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.641720 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.641821 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.642036 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.644799 (+ 2763us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.644979 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30502,"thread_start_us":31351,"threads_started":1}
I20260806 01:10:16.645867 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.611615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.22.63.66:44407)
0806 01:10:16.627599 (+ 15984us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.627609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.629346 (+ 1737us) server_negotiation.cc:407] Connection header received
0806 01:10:16.629458 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.629466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.629722 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.629955 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:16.631129 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.633430 (+ 2301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.638788 (+ 5358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.641623 (+ 2835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.641780 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.642012 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.643763 (+ 1751us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.645527 (+ 1764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15796,"thread_start_us":15980,"threads_started":1}
I20260806 01:10:16.646510 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.580734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57744)
0806 01:10:16.609683 (+ 28949us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.609767 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.622754 (+ 12987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.629678 (+ 6924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.629687 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.629720 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:16.630851 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.630862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.638018 (+ 7156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.638024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.640063 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.640079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.642234 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.642239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.644572 (+ 2333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.644604 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.645947 (+ 1343us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.646123 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28519,"mutex_wait_us":2118,"thread_start_us":40694,"threads_started":1}
I20260806 01:10:16.647881 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.583901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.22.63.67:45783)
0806 01:10:16.611078 (+ 27177us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.611088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.621117 (+ 10029us) server_negotiation.cc:407] Connection header received
0806 01:10:16.621492 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.621503 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.621857 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.622093 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:16.631280 (+ 9187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.633174 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.642447 (+ 9273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.644014 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.644150 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.644398 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.647342 (+ 2944us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.647528 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26955,"thread_start_us":27192,"threads_started":1}
I20260806 01:10:16.650777 10349 tablet_service.cc:1467] Processing CreateTablet for tablet 96d3c47b26be48db95375aade57e7a16 (DEFAULT_TABLE table=test-table [id=def353a8c2ad477d970aee572fba7764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:16.648833 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.583540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:45783 (local address 127.0.0.1:43418)
0806 01:10:16.620955 (+ 37415us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.620980 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.621233 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.630187 (+ 8954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.630192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.630217 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:16.631025 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.631037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.633389 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.633394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.636001 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.636020 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.646929 (+ 10909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.646936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.647114 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.647151 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.648297 (+ 1146us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.648461 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37246,"mutex_wait_us":342,"thread_start_us":26486,"threads_started":1}
I20260806 01:10:16.653409 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d3c47b26be48db95375aade57e7a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:16.655287 10087 tablet_service.cc:1467] Processing CreateTablet for tablet 96d3c47b26be48db95375aade57e7a16 (DEFAULT_TABLE table=test-table [id=def353a8c2ad477d970aee572fba7764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:16.656996 10087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d3c47b26be48db95375aade57e7a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:16.663522 10559 tablet_service.cc:1467] Processing CreateTablet for tablet 96d3c47b26be48db95375aade57e7a16 (DEFAULT_TABLE table=test-table [id=def353a8c2ad477d970aee572fba7764]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:16.665050 10559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d3c47b26be48db95375aade57e7a16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:16.684592 10698 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Bootstrap starting.
I20260806 01:10:16.689105 10698 tablet_bootstrap.cc:654] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:16.690703 10698 log.cc:824] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:16.690780 10700 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Bootstrap starting.
I20260806 01:10:16.699046 10702 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Bootstrap starting.
I20260806 01:10:16.705711 10700 tablet_bootstrap.cc:654] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:16.705709 10702 tablet_bootstrap.cc:654] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:16.707188 10702 log.cc:824] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:16.707188 10700 log.cc:824] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:16.713451 10698 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: No bootstrap required, opened a new log
I20260806 01:10:16.713843 10698 ts_tablet_manager.cc:1397] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Time spent bootstrapping tablet: real 0.030s user 0.012s sys 0.000s
I20260806 01:10:16.722478 10702 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: No bootstrap required, opened a new log
I20260806 01:10:16.722847 10702 ts_tablet_manager.cc:1397] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.012s
I20260806 01:10:16.723717 10700 tablet_bootstrap.cc:492] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: No bootstrap required, opened a new log
I20260806 01:10:16.724062 10700 ts_tablet_manager.cc:1397] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.011s
I20260806 01:10:16.725029 10698 raft_consensus.cc:348] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.725621 10698 raft_consensus.cc:374] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:16.725770 10698 raft_consensus.cc:729] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65cb671405094036aa280533ed0aaeab, State: Initialized, Role: FOLLOWER
I20260806 01:10:16.726583 10698 consensus_queue.cc:260] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.735213 10698 ts_tablet_manager.cc:1428] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Time spent starting tablet: real 0.021s user 0.010s sys 0.010s
I20260806 01:10:16.735173 10702 raft_consensus.cc:348] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.735672 10450 heartbeater.cc:503] Master 127.22.63.126:41999 was elected leader, sending a full tablet report...
I20260806 01:10:16.735708 10702 raft_consensus.cc:374] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:16.735831 10702 raft_consensus.cc:729] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd25f8e463c4c919733f108a835626b, State: Initialized, Role: FOLLOWER
I20260806 01:10:16.736557 10702 consensus_queue.cc:260] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.741875 10700 raft_consensus.cc:348] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.742393 10700 raft_consensus.cc:374] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:16.742522 10700 raft_consensus.cc:729] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ce867165ff43aa9a6881c85d570e28, State: Initialized, Role: FOLLOWER
I20260806 01:10:16.743204 10700 consensus_queue.cc:260] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.743551 10661 heartbeater.cc:503] Master 127.22.63.126:41999 was elected leader, sending a full tablet report...
I20260806 01:10:16.744526 10702 ts_tablet_manager.cc:1428] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Time spent starting tablet: real 0.022s user 0.011s sys 0.008s
I20260806 01:10:16.745677 10216 heartbeater.cc:503] Master 127.22.63.126:41999 was elected leader, sending a full tablet report...
I20260806 01:10:16.750802 10700 ts_tablet_manager.cc:1428] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Time spent starting tablet: real 0.027s user 0.012s sys 0.002s
W20260806 01:10:16.803934 10217 tablet.cc:2369] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:16.827106 10712 raft_consensus.cc:482] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:10:16.827467 10712 raft_consensus.cc:504] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.829828 10712 leader_election.cc:288] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50ce867165ff43aa9a6881c85d570e28 (127.22.63.65:38997), 65cb671405094036aa280533ed0aaeab (127.22.63.66:44407)
I20260806 01:10:16.836818 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.830872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:55217 (local address 127.22.63.66:44407)
0806 01:10:16.831340 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.831345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.831359 (+ 14us) server_negotiation.cc:407] Connection header received
0806 01:10:16.831417 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.831420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.831482 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.831591 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:16.832361 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.833079 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.834943 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.835888 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.836043 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.836124 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.836401 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.836532 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344}
I20260806 01:10:16.836896 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.830207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:44407 (local address 127.22.63.67:55217)
0806 01:10:16.831084 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.831104 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.831247 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.831613 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.831618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.831646 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:16.832190 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.832204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.833222 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.833228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.834789 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.834803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.836065 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.836070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.836222 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.836254 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.836360 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.836499 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":115,"spinlock_wait_cycles":11520,"thread_start_us":722,"threads_started":1}
I20260806 01:10:16.838109 10664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.830096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.22.63.67:37775)
0806 01:10:16.830452 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:16.830475 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:16.830611 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:16.831745 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:16.831749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:16.831841 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:16.832248 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.832263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.833773 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.833778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.835767 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:16.835782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.837361 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.837367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:16.837541 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.837574 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:10:16.837670 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 01:10:16.837804 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":53,"spinlock_wait_cycles":31360}
I20260806 01:10:16.838093 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d3c47b26be48db95375aade57e7a16" candidate_uuid: "0fd25f8e463c4c919733f108a835626b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cb671405094036aa280533ed0aaeab" is_pre_election: true
I20260806 01:10:16.838744 10379 raft_consensus.cc:2454] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd25f8e463c4c919733f108a835626b in term 0.
I20260806 01:10:16.838832 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.830266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:37775 (local address 127.22.63.65:38997)
0806 01:10:16.830502 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.830508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.830533 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:10:16.830880 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.830886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.830977 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.831069 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:16.832464 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.833587 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.835961 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.837137 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.837397 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.837478 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.838446 (+ 968us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.838591 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:10:16.839524 10480 leader_election.cc:302] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd25f8e463c4c919733f108a835626b, 65cb671405094036aa280533ed0aaeab; no voters:
I20260806 01:10:16.840018 10712 raft_consensus.cc:2781] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:10:16.839879 10129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d3c47b26be48db95375aade57e7a16" candidate_uuid: "0fd25f8e463c4c919733f108a835626b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ce867165ff43aa9a6881c85d570e28" is_pre_election: true
I20260806 01:10:16.840178 10712 raft_consensus.cc:482] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:10:16.840282 10712 raft_consensus.cc:3037] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:16.840430 10129 raft_consensus.cc:2454] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fd25f8e463c4c919733f108a835626b in term 0.
I20260806 01:10:16.843762 10712 raft_consensus.cc:504] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.844506 10712 leader_election.cc:288] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [CANDIDATE]: Term 1 election: Requested vote from peers 50ce867165ff43aa9a6881c85d570e28 (127.22.63.65:38997), 65cb671405094036aa280533ed0aaeab (127.22.63.66:44407)
I20260806 01:10:16.844985 10129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d3c47b26be48db95375aade57e7a16" candidate_uuid: "0fd25f8e463c4c919733f108a835626b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ce867165ff43aa9a6881c85d570e28"
I20260806 01:10:16.845202 10129 raft_consensus.cc:3037] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:16.845194 10379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d3c47b26be48db95375aade57e7a16" candidate_uuid: "0fd25f8e463c4c919733f108a835626b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65cb671405094036aa280533ed0aaeab"
I20260806 01:10:16.845419 10379 raft_consensus.cc:3037] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:16.847677 10379 raft_consensus.cc:2454] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd25f8e463c4c919733f108a835626b in term 1.
I20260806 01:10:16.848161 10480 leader_election.cc:302] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd25f8e463c4c919733f108a835626b, 65cb671405094036aa280533ed0aaeab; no voters:
I20260806 01:10:16.848667 10129 raft_consensus.cc:2454] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fd25f8e463c4c919733f108a835626b in term 1.
I20260806 01:10:16.849043 10712 raft_consensus.cc:2781] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:16.849557 10712 raft_consensus.cc:686] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 1 LEADER]: Becoming Leader. State: Replica: 0fd25f8e463c4c919733f108a835626b, State: Running, Role: LEADER
I20260806 01:10:16.850148 10712 consensus_queue.cc:237] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } }
I20260806 01:10:16.856386 9875 catalog_manager.cc:5495] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fd25f8e463c4c919733f108a835626b (127.22.63.67). New cstate: current_term: 1 leader_uuid: "0fd25f8e463c4c919733f108a835626b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fd25f8e463c4c919733f108a835626b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 45783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 } health_report { overall_health: UNKNOWN } } }
I20260806 01:10:16.903195 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.897804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.22.63.67:45783)
0806 01:10:16.898061 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.898070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.898091 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:10:16.898195 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.898199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.898283 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.898451 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:10:16.898860 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.899582 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.901032 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.901439 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.901573 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.902100 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:10:16.902616 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:10:16.902791 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.902844 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.902925 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:10:16.911944 10129 raft_consensus.cc:1261] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 1 FOLLOWER]: Refusing update from remote peer 0fd25f8e463c4c919733f108a835626b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:16.912140 10379 raft_consensus.cc:1261] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 1 FOLLOWER]: Refusing update from remote peer 0fd25f8e463c4c919733f108a835626b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:16.913087 10727 consensus_queue.cc:1035] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ce867165ff43aa9a6881c85d570e28" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 38997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:16.913503 10712 consensus_queue.cc:1035] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [LEADER]: Connected to new peer: Peer: permanent_uuid: "65cb671405094036aa280533ed0aaeab" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:16.930960 10737 mvcc.cc:205] Tried to move back new op lower bound from 7315368414859464704 to 7315368414623420416. Current Snapshot: MvccSnapshot[applied={T|T < 7315368414859464704}]
I20260806 01:10:16.947697 9875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:16.948060 9875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:16.952852 9875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a17c3e8d4fc64441b8c8d2d26591d9bf: Sending DeleteTablet for 3 replicas of tablet 96d3c47b26be48db95375aade57e7a16
I20260806 01:10:16.954600 10559 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d3c47b26be48db95375aade57e7a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:16 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:43418
I20260806 01:10:16.954833 10087 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d3c47b26be48db95375aade57e7a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:16 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:57744
I20260806 01:10:16.954862 10083 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d3c47b26be48db95375aade57e7a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:16 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:57744
I20260806 01:10:16.955555 10349 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d3c47b26be48db95375aade57e7a16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:16 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:43246
I20260806 01:10:16.956489 10750 tablet_replica.cc:330] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: stopping tablet replica
I20260806 01:10:16.956955 10750 raft_consensus.cc:2229] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:16.957566 10750 raft_consensus.cc:2258] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:16.962090 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 9933
I20260806 01:10:16.965494 10750 ts_tablet_manager.cc:1905] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:16.971145 10747 tablet_replica.cc:330] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: stopping tablet replica
I20260806 01:10:16.973348 10750 ts_tablet_manager.cc:1918] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:16.973657 10750 log.cc:1196] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/96d3c47b26be48db95375aade57e7a16
I20260806 01:10:16.974334 10750 ts_tablet_manager.cc:1939] T 96d3c47b26be48db95375aade57e7a16 P 65cb671405094036aa280533ed0aaeab: Deleting consensus metadata
I20260806 01:10:16.976918 9856 catalog_manager.cc:4841] TS 65cb671405094036aa280533ed0aaeab (127.22.63.66:44407): tablet 96d3c47b26be48db95375aade57e7a16 (table test-table [id=def353a8c2ad477d970aee572fba7764]) successfully deleted
I20260806 01:10:16.971663 10747 raft_consensus.cc:2229] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:10:16.977810 10747 raft_consensus.cc:2258] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:16.980062 10747 ts_tablet_manager.cc:1905] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:16.984072 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:16.964239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.22.63.65:38997)
0806 01:10:16.964431 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:16.964437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:16.964473 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:10:16.964543 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:16.964548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:16.964653 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:16.964774 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:16.966795 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.967984 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.969906 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:16.970565 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:16.970688 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:16.976192 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:16.976211 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:16.976229 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:16.976284 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:16.979646 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.980906 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.980913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.980916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.981052 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:16.981643 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:16.981649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:16.981652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:16.982035 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:16.982381 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:16.983314 (+ 933us) server_negotiation.cc:299] Negotiation successful
0806 01:10:16.983558 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:10:16.984864 10208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:16.986963 10747 ts_tablet_manager.cc:1918] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:16.987144 10747 log.cc:1196] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/96d3c47b26be48db95375aade57e7a16
I20260806 01:10:16.987725 10747 ts_tablet_manager.cc:1939] T 96d3c47b26be48db95375aade57e7a16 P 0fd25f8e463c4c919733f108a835626b: Deleting consensus metadata
I20260806 01:10:16.997406 9856 catalog_manager.cc:4841] TS 0fd25f8e463c4c919733f108a835626b (127.22.63.67:45783): tablet 96d3c47b26be48db95375aade57e7a16 (table test-table [id=def353a8c2ad477d970aee572fba7764]) successfully deleted
I20260806 01:10:17.090533 10748 tablet_replica.cc:330] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: stopping tablet replica
I20260806 01:10:17.091046 10748 raft_consensus.cc:2229] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:17.091563 10748 raft_consensus.cc:2258] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:17.094035 10748 ts_tablet_manager.cc:1905] T 96d3c47b26be48db95375aade57e7a16 P 50ce867165ff43aa9a6881c85d570e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:17.100338 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 9933
W20260806 01:10:17.127900 9855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:38997 (error 108)
I20260806 01:10:17.128863 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 10222
I20260806 01:10:17.141812 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.141301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57774)
0806 01:10:17.141474 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.141618 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 01:10:17.142130 9855 catalog_manager.cc:4584] TS 50ce867165ff43aa9a6881c85d570e28 (127.22.63.65:38997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96d3c47b26be48db95375aade57e7a16: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
I20260806 01:10:17.150718 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.131154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.22.63.66:44407)
0806 01:10:17.131385 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:17.131393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:17.131419 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:10:17.131518 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:17.131526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:17.131630 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:17.134152 (+ 2522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:17.134384 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:17.135875 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:17.138587 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:17.139512 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:17.139785 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:17.144208 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:17.144222 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:17.144239 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:17.144277 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:17.147512 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:17.148363 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:17.148371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:17.148373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:17.148516 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:17.148945 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:17.148949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:17.148951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:17.149453 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:17.149769 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:17.150095 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 01:10:17.150322 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:10:17.151508 10443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:17.186971 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.186297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57786)
0806 01:10:17.186668 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.186788 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260806 01:10:17.188277 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.187876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57796)
0806 01:10:17.188060 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.188145 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 01:10:17.195762 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.195262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57802)
0806 01:10:17.195487 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.195568 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 01:10:17.196869 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.196497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:57816)
0806 01:10:17.196677 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.196748 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 01:10:17.237829 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.237211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55456)
0806 01:10:17.237507 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.237606 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260806 01:10:17.239130 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.238681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55458)
0806 01:10:17.238923 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.239000 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 01:10:17.260989 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 10222
I20260806 01:10:17.273926 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.273240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55472)
0806 01:10:17.273574 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.273686 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260806 01:10:17.275262 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.274748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55484)
0806 01:10:17.274990 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.275100 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260806 01:10:17.287272 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 10454
I20260806 01:10:17.303012 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.288088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.22.63.67:45783)
0806 01:10:17.288278 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:17.288285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:17.288444 (+ 159us) server_negotiation.cc:407] Connection header received
0806 01:10:17.288677 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:17.288682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:17.288754 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:17.288868 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:17.290329 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:17.291157 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:17.292706 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:17.293177 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:17.293333 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:17.296390 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:17.296404 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:17.296421 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:17.296467 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:17.299727 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:17.300674 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:17.300682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:17.300684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:17.300831 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:17.301254 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:17.301259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:17.301263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:17.301671 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:17.302039 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:17.302368 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 01:10:17.302606 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:10:17.303925 10653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:17.340065 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.339520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55498)
0806 01:10:17.339769 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.339873 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:10:17.341514 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.340994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55502)
0806 01:10:17.341173 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.341337 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 01:10:17.377125 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.376615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55516)
0806 01:10:17.376846 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.376948 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 01:10:17.378296 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.377955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55532)
0806 01:10:17.378103 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.378176 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 01:10:17.422065 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 10454
I20260806 01:10:17.450513 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 9830
I20260806 01:10:17.452116 9919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:17.565119 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:17.564559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:38997 (local address 127.0.0.1:55534)
0806 01:10:17.564805 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:17.564903 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:38997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260806 01:10:17.565141 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 9830
I20260806 01:10:17.600174 22781 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 01:10:17.946357 22781 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 01:10:17.946619 22781 mini_ranger.cc:66] Stopping Ranger...
I20260806 01:10:18.304982 22781 mini_ranger.cc:68] Stopped Ranger
2026-08-06 01:10:18.305 UTC [8217] LOG: received smart shutdown request
2026-08-06 01:10:18.308 UTC [8217] LOG: background worker "logical replication launcher" (PID 8232) exited with exit code 1
2026-08-06 01:10:18.308 UTC [8224] LOG: shutting down
2026-08-06 01:10:18.482 UTC [8217] LOG: database system is shut down
2026-08-06T01:10:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54874 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:10:18 dist-test-slave-7f01 krb5kdc[10769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:10:18 dist-test-slave-7f01 krb5kdc[10769](info): set up 2 sockets
Aug 06 01:10:18 dist-test-slave-7f01 krb5kdc[10769](info): commencing operation
krb5kdc: starting...
W20260806 01:10:19.965232 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:10:20 dist-test-slave-7f01 krb5kdc[10769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:10:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:10:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:10:20.263883 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:34597
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:20.623582 10786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:20.623924 10786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:20.624106 10786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:20.631713 10786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:20.631861 10786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:20.631979 10786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:20.632097 10786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:20.632179 10786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:20.642427 10786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:34597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:20.644012 10786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:20.645395 10786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:20.658020 10794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:20.658052 10795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:20.658108 10800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:20.659153 10786 server_base.cc:1034] running on GCE node
I20260806 01:10:20.660051 10786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:20.661693 10786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:20.662966 10786 hybrid_clock.cc:648] HybridClock initialized: now 1785978620662923 us; error 63 us; skew 500 ppm
I20260806 01:10:20.663379 10786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:20 dist-test-slave-7f01 krb5kdc[10769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:20.671114 10786 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:20.672864 10786 webserver.cc:460] Webserver started at http://127.22.63.126:33259/ using document root <none> and password file <none>
I20260806 01:10:20.673662 10786 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:20.673818 10786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:20.674150 10786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:20.677477 10786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27"
format_stamp: "Formatted at 2026-08-06 01:10:20 on dist-test-slave-7f01"
server_key: "32f1f3b716215f55f2d3e84c5d435388"
server_key_iv: "945d420bdff9858b023a77f0c71eade7"
server_key_version: "encryptionkey@0"
I20260806 01:10:20.678364 10786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27"
format_stamp: "Formatted at 2026-08-06 01:10:20 on dist-test-slave-7f01"
server_key: "32f1f3b716215f55f2d3e84c5d435388"
server_key_iv: "945d420bdff9858b023a77f0c71eade7"
server_key_version: "encryptionkey@0"
I20260806 01:10:20.684002 10786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 01:10:20.687708 10814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:20.689206 10786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:20.689523 10786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27"
format_stamp: "Formatted at 2026-08-06 01:10:20 on dist-test-slave-7f01"
server_key: "32f1f3b716215f55f2d3e84c5d435388"
server_key_iv: "945d420bdff9858b023a77f0c71eade7"
server_key_version: "encryptionkey@0"
I20260806 01:10:20.689743 10786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:20.718734 10786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:20.723970 10786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:20.724316 10786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:20.745465 10786 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:34597
I20260806 01:10:20.745502 10891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:34597 every 8 connection(s)
I20260806 01:10:20.746995 10786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:20.751057 10892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:20.755395 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 10786
I20260806 01:10:20.755637 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:10:20.756488 22781 external_mini_cluster.cc:1428] Setting key 18dbd99d3c0b757fd8f9c266776979a2
I20260806 01:10:20.759416 10892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:10:20 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.63.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.125@KRBTEST.COM" created.
I20260806 01:10:20.792456 10894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.763609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.22.63.126:34597)
0806 01:10:20.770726 (+ 7117us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:20.770737 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:20.770799 (+ 62us) server_negotiation.cc:407] Connection header received
0806 01:10:20.772349 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:20.772360 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:20.772672 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:20.772897 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:20.774150 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:20.776298 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:20.777775 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:20.778743 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:20.778954 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:20.782534 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:20.782548 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:20.782568 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:20.782613 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:20.787202 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:20.788185 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:20.788192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:20.788194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:20.788370 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:20.788956 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:20.788960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:20.788962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:20.789574 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:20.789779 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:20.790554 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 01:10:20.790910 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":6861,"thread_start_us":6586,"threads_started":1}
I20260806 01:10:20.792493 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.763609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34597 (local address 127.0.0.1:43766)
0806 01:10:20.769593 (+ 5984us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.769652 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:20.771952 (+ 2300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:20.772986 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:20.772992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:20.773584 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:20.773934 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:20.773951 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:20.776489 (+ 2538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:20.776493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:20.777615 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:20.777628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:20.779034 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:20.779039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:20.779219 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:20.779993 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:20.780017 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:20.782254 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:20.787399 (+ 5145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:20.787410 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:20.787421 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:20.788046 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:20.788590 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:20.788597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:20.788604 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:20.788808 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:20.789751 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:20.789757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:20.789993 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 01:10:20.790415 (+ 422us) client_negotiation.cc:240] Negotiation successful
0806 01:10:20.791019 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5636,"mutex_wait_us":152,"spinlock_wait_cycles":42880,"thread_start_us":5339,"threads_started":1}
I20260806 01:10:20.795899 10892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:10:20.797254 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.796701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50896)
0806 01:10:20.796936 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.797081 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 01:10:20.797576 10818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
I20260806 01:10:20.799353 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.798884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50904)
0806 01:10:20.799081 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.799177 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 01:10:20.799590 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:33403: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
I20260806 01:10:20.826809 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:33403
--webserver_interface=127.22.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:20.849040 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } attempt: 1
I20260806 01:10:20.850574 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.849886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50910)
0806 01:10:20.850164 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.850366 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:10:20.851900 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.851365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50920)
0806 01:10:20.851575 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.851660 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 01:10:20.852185 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:33403: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
I20260806 01:10:20.920477 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } attempt: 2
I20260806 01:10:20.921756 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.921234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50924)
0806 01:10:20.921522 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.921619 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:10:20.923036 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:20.922545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50936)
0806 01:10:20.922835 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:20.922903 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260806 01:10:20.923239 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:33403: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
I20260806 01:10:21.014544 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } attempt: 3
I20260806 01:10:21.015866 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.015328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50950)
0806 01:10:21.015548 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.015654 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 01:10:21.017040 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.016618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50954)
0806 01:10:21.016776 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.016870 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 01:10:21.017234 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:33403: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
I20260806 01:10:21.160512 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } attempt: 4
I20260806 01:10:21.161970 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.161241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50964)
0806 01:10:21.161643 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.161778 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260806 01:10:21.163252 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.162809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50972)
0806 01:10:21.162982 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.163087 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 01:10:21.163488 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:33403: Network error: Client connection negotiation failed: client connection to 127.22.63.125:33403: connect: Connection refused (error 111)
W20260806 01:10:21.186518 10904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:21.186985 10904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:21.187165 10904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:21.194885 10904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:21.195050 10904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:21.195163 10904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:21.195266 10904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:21.195351 10904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:21.205559 10904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:33403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:21.207150 10904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:21.208580 10904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:21.220911 10914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:21.220952 10910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:21.220999 10909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:21.225188 10904 server_base.cc:1034] running on GCE node
I20260806 01:10:21.226055 10904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:21.227674 10904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:21.228924 10904 hybrid_clock.cc:648] HybridClock initialized: now 1785978621228867 us; error 55 us; skew 500 ppm
I20260806 01:10:21.229308 10904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:21.232687 10904 init.cc:377] Logged in from keytab as kudu/127.22.63.125@KRBTEST.COM (short username kudu)
I20260806 01:10:21.234320 10904 webserver.cc:460] Webserver started at http://127.22.63.125:39459/ using document root <none> and password file <none>
I20260806 01:10:21.235042 10904 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:21.235146 10904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:21.235476 10904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:21.238703 10904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data/instance:
uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "17968619d29bffa52c5b91b8d72b5511"
server_key_iv: "73cdd93275cd92b874b39ec68032db78"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.239481 10904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal/instance:
uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "17968619d29bffa52c5b91b8d72b5511"
server_key_iv: "73cdd93275cd92b874b39ec68032db78"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.244861 10904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:10:21.248381 10927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:21.249745 10904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:21.249946 10904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "17968619d29bffa52c5b91b8d72b5511"
server_key_iv: "73cdd93275cd92b874b39ec68032db78"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.250166 10904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:21.262671 10904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:21.267202 10904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:21.267566 10904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:21.287475 10904 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.125:33403
I20260806 01:10:21.287508 11012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.125:33403 every 8 connection(s)
I20260806 01:10:21.289044 10904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
I20260806 01:10:21.293010 11013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:21.299045 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 10904
I20260806 01:10:21.299285 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-1/wal/instance
I20260806 01:10:21.299808 22781 external_mini_cluster.cc:1428] Setting key 3dbcac33f8b1d58f0671bb92fd017f3b
I20260806 01:10:21.299976 11013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:21.321233 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.302647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.22.63.126:34597)
0806 01:10:21.303189 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.303194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.304492 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 01:10:21.306433 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.306439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.306532 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.306664 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.308350 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.309110 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.310694 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.311139 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.311288 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.315103 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.315108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.315111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.315156 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.317963 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.318752 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.318757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.318760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.318885 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.319358 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.319363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.319365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.319652 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.319851 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.320611 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.320843 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":317,"threads_started":1}
I20260806 01:10:21.322254 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.303276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34597 (local address 127.0.0.1:43780)
0806 01:10:21.304304 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.304344 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.306140 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.306931 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.306939 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.307473 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.308147 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.308166 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.309288 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.309292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.310540 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.310552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.311380 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.311388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.311628 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.312399 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.312422 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.314833 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.318129 (+ 3296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.318138 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.318157 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.318611 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.319008 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.319013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.319017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.319226 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.319830 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.319838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.320230 (+ 392us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.320502 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.321046 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":559,"threads_started":1}
I20260806 01:10:21.324219 11013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.124@KRBTEST.COM" created.
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:10:21.341009 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.325465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.22.63.125:33403)
0806 01:10:21.326203 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.326209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.326234 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:10:21.326278 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.326284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.326452 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.326610 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.327361 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.328511 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.329681 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.330219 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.330357 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.334007 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.334019 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.334034 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.334062 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.337683 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.338586 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.338597 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.338603 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.338740 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.339194 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.339201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.339206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.339666 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.339893 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.340346 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.340675 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":598,"thread_start_us":457,"threads_started":1}
I20260806 01:10:21.341022 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.325223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50980)
0806 01:10:21.325440 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.325462 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.325559 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.326596 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.326600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.327060 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.327198 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.327206 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.328682 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.328692 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.329536 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.329547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.330417 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.330426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.330633 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.331343 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.331361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.333775 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.337851 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.337860 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.337866 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.338447 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.338857 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.338865 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.338871 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.339082 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.339824 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.339836 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.340025 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.340192 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.340551 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:10:21.343497 11013 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:10:21.344820 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.344333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58010)
0806 01:10:21.344539 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.344665 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 01:10:21.345120 10932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.346843 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.346475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58026)
0806 01:10:21.346638 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.346721 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 01:10:21.347095 11013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
I20260806 01:10:21.367297 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:43659
--webserver_interface=127.22.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:45951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:21.396418 11013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 1
I20260806 01:10:21.397724 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.397138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58036)
0806 01:10:21.397391 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.397527 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:10:21.398795 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.398422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58044)
0806 01:10:21.398586 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.398669 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:10:21.399036 11013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.462759 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } attempt: 5
I20260806 01:10:21.467263 11013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 2
I20260806 01:10:21.468432 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.467895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58052)
0806 01:10:21.468110 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.468219 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 01:10:21.469597 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.469199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58058)
0806 01:10:21.469350 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.469443 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 01:10:21.469918 11013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:10:21.478041 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.463778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50982 (local address 127.22.63.125:33403)
0806 01:10:21.464010 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.464019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.464050 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:10:21.464148 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.464156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.464252 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.464413 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.465317 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.466202 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.467424 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.468182 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.468322 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.472350 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.472356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.472359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.472395 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.475164 (+ 2769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.476022 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.476026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.476029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.476158 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.476572 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.476575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.476577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.476862 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.477036 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.477464 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.477687 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:10:21.478041 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.463629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50982)
0806 01:10:21.463856 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.463878 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.464004 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.464424 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.464428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.464948 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.465138 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.465148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.466349 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.466355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.467268 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.467281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.468339 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.468345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.468553 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.469329 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.469353 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.472117 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.475328 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.475334 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.475338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.475907 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.476272 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.476276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.476280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.476461 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.477006 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.477012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.477181 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.477335 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.477681 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:10:21.479501 10892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:10:21.480700 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.480293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58068)
0806 01:10:21.480478 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.480567 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:10:21.481640 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.481310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58082)
0806 01:10:21.481461 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.481548 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:10:21.481873 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.533133 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 1
I20260806 01:10:21.534475 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.533854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58088)
0806 01:10:21.534116 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.534240 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 01:10:21.535717 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.535233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58092)
0806 01:10:21.535435 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.535553 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 01:10:21.536041 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.561187 11013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 3
I20260806 01:10:21.562420 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.561904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58098)
0806 01:10:21.562140 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.562230 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:10:21.563689 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.563210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58102)
0806 01:10:21.563446 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.563524 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260806 01:10:21.563992 11013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.604305 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 2
I20260806 01:10:21.605697 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.605044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58118)
0806 01:10:21.605297 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.605451 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:10:21.606923 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.606432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58122)
0806 01:10:21.606624 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.606732 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 01:10:21.607146 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.707247 11013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 4
I20260806 01:10:21.708523 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.708011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58132)
0806 01:10:21.708270 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.708359 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 01:10:21.709515 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.709201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58148)
0806 01:10:21.709343 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.709419 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 01:10:21.709720 11013 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
I20260806 01:10:21.713397 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 3
I20260806 01:10:21.714524 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.714000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58158)
0806 01:10:21.714218 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.714322 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:10:21.715672 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.715186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58162)
0806 01:10:21.715417 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.715534 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260806 01:10:21.715956 10892 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:43659: Network error: Client connection negotiation failed: client connection to 127.22.63.124:43659: connect: Connection refused (error 111)
W20260806 01:10:21.726121 11021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:21.726444 11021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:21.726605 11021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:21.734323 11021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:21.734455 11021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:21.734543 11021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:21.734639 11021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:21.734720 11021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:21.744957 11021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:34597,127.22.63.125:33403,127.22.63.124:43659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:21.746701 11021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:21.748113 11021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:21.761296 11032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:21.761339 11033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:21.761405 11036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:21.762364 11021 server_base.cc:1034] running on GCE node
I20260806 01:10:21.763236 11021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:21.764779 11021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:21.765996 11021 hybrid_clock.cc:648] HybridClock initialized: now 1785978621765945 us; error 42 us; skew 500 ppm
I20260806 01:10:21.766392 11021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:21.769630 11021 init.cc:377] Logged in from keytab as kudu/127.22.63.124@KRBTEST.COM (short username kudu)
I20260806 01:10:21.771189 11021 webserver.cc:460] Webserver started at http://127.22.63.124:32911/ using document root <none> and password file <none>
I20260806 01:10:21.772006 11021 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:21.772161 11021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:21.772473 11021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:21.775504 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data/instance:
uuid: "64d349195eeb48a882739d8eac47fea2"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "23604b3c177e28209033266b3522739b"
server_key_iv: "b0ceac5c560fb94c101833c3d76cb0ea"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.776408 11021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal/instance:
uuid: "64d349195eeb48a882739d8eac47fea2"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "23604b3c177e28209033266b3522739b"
server_key_iv: "b0ceac5c560fb94c101833c3d76cb0ea"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.781641 11021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:10:21.785205 11051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:21.786604 11021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:21.786846 11021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "64d349195eeb48a882739d8eac47fea2"
format_stamp: "Formatted at 2026-08-06 01:10:21 on dist-test-slave-7f01"
server_key: "23604b3c177e28209033266b3522739b"
server_key_iv: "b0ceac5c560fb94c101833c3d76cb0ea"
server_key_version: "encryptionkey@0"
I20260806 01:10:21.787098 11021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:21.809222 11021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:21.813561 11021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:21.813884 11021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:21.833420 11021 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.124:43659
I20260806 01:10:21.833451 11139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.124:43659 every 8 connection(s)
I20260806 01:10:21.834930 11021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
I20260806 01:10:21.838181 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 11021
I20260806 01:10:21.838418 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785978402713339-22781-0/minicluster-data/master-2/wal/instance
I20260806 01:10:21.839147 22781 external_mini_cluster.cc:1428] Setting key 094a61163d54020aba190c411f0859b1
I20260806 01:10:21.839280 11140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:21.850257 11140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:21.864863 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.841268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.22.63.124:43659)
0806 01:10:21.842137 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.842149 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.842195 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:10:21.844871 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.844902 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.845452 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.845902 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.847508 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.849516 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.850908 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.851913 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.852103 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.857398 (+ 5295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.857406 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.857416 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.857460 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.860593 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.861374 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.861382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.861388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.861564 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.861966 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.861971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.861974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.862575 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.862890 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.863353 (+ 463us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.863838 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":433,"threads_started":1}
I20260806 01:10:21.869112 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.853282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34597 (local address 127.0.0.1:43792)
0806 01:10:21.854175 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.854210 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.854390 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.854908 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.854917 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.855848 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.856149 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.856158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.857248 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.857253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.858584 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.858599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.859591 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.859600 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.859858 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.860508 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.860526 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.862605 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.865939 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.865952 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.865959 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.866568 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.866959 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.866963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.866966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.867133 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.867836 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.867845 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.868021 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.868219 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.868591 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":489,"threads_started":1}
I20260806 01:10:21.869189 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.853222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43792 (local address 127.22.63.126:34597)
0806 01:10:21.853807 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.853814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.854305 (+ 491us) server_negotiation.cc:407] Connection header received
0806 01:10:21.854537 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.854541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.854651 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.854784 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.856315 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.857081 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.858750 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.859370 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.859516 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.862827 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.862833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.862837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.862881 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.865802 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.866693 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.866697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.866700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.866840 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.867257 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.867262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.867265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.867644 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.867825 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.868386 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.868642 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":284,"threads_started":1}
I20260806 01:10:21.870795 11140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:10:21.888927 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.871594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:33403 (local address 127.0.0.1:50986)
0806 01:10:21.871851 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.871873 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.871975 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.872293 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.872297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.872735 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.872980 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.872995 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.874407 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.874415 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.875847 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.875864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.877118 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.877126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.877392 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.878058 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.878086 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.881585 (+ 3499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.885743 (+ 4158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.885751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.885757 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.886575 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.887142 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.887147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.887150 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.887335 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.887882 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.887887 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.888066 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.888214 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.888528 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 01:10:21.889063 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.868259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.22.63.126:34597)
0806 01:10:21.869299 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.869306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.869331 (+ 25us) server_negotiation.cc:407] Connection header received
0806 01:10:21.869381 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.869386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.869454 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.869588 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.871552 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.872893 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.874716 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.875405 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.875536 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.881574 (+ 6038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.881581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.881586 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.881630 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.885450 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.886549 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.886560 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.886566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.886791 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.887382 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.887386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.887388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.887703 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.888051 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.888399 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.888648 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":568,"threads_started":1}
I20260806 01:10:21.889171 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.871674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.22.63.125:33403)
0806 01:10:21.871912 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.871917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.871934 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:10:21.872092 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.872096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.872156 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.872235 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.873158 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.874215 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.876014 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.876736 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.876888 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.881866 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.881873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.881880 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.881929 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.885571 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.886726 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.886788 (+ 62us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.886794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.886983 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.887443 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.887449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.887452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.887728 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.887924 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.888333 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.888558 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 01:10:21.890055 11140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:10:21.893211 10892 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 4
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
Aug 06 01:10:21 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978620, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:10:21.909574 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.890830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.22.63.125:33403)
0806 01:10:21.891347 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.891351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.891364 (+ 13us) server_negotiation.cc:407] Connection header received
0806 01:10:21.891400 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.891404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.891456 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.891561 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.893285 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.894335 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.896142 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.896591 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.896731 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.902264 (+ 5533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.902270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.902275 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.902322 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.906008 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.907013 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.907019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.907023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.907195 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.907706 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.907713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.907720 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.908213 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.908496 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.908837 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.909110 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389}
I20260806 01:10:21.910559 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.890922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58170)
0806 01:10:21.891819 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.891838 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.891949 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.892252 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.892257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.892780 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.892968 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.892981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.894292 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.894297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.895925 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.895941 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.897111 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.897116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.897317 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.898906 (+ 1589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.898927 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.902919 (+ 3992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.907450 (+ 4531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.907455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.907459 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.908057 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.908507 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.908514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.908520 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.908781 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.909487 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.909493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.909641 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.909788 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.910119 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"spinlock_wait_cycles":32000}
I20260806 01:10:21.910598 11143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.891143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58170 (local address 127.22.63.124:43659)
0806 01:10:21.891302 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.891309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.891907 (+ 598us) server_negotiation.cc:407] Connection header received
0806 01:10:21.892051 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.892054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.892109 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.892185 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.893144 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.894158 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.896081 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.896800 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.896914 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.903070 (+ 6156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.903076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.903080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.903143 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.907329 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.908197 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.908200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.908202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.908375 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.908889 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.908894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.908896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.909302 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.909423 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.909883 (+ 460us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.910287 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":36}
I20260806 01:10:21.911106 10896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.893796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58172)
0806 01:10:21.893977 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:21.893994 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:21.894087 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:21.895337 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:21.895344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:21.895829 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:21.896023 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.896038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.897703 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.897708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.898620 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:21.898632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.899649 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.899658 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:21.899910 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.900578 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:21.900602 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:21.903270 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:21.907032 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.907037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.907041 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.907564 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.908868 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:21.908876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:21.908879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:21.909070 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:21.909834 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:21.909838 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:21.910035 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 01:10:21.910195 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 01:10:21.910524 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 01:10:21.912101 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.894428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.22.63.124:43659)
0806 01:10:21.895072 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.895076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.895093 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:10:21.895130 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.895133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.895183 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.895312 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.896207 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.897511 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.898767 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.899446 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.899584 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.903515 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.903520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.903523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.903555 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.906871 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.908550 (+ 1679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.908556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.908562 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.908701 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.909226 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.909231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.909234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.909563 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.909701 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.911407 (+ 1706us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.911593 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":549,"thread_start_us":345,"threads_started":1}
I20260806 01:10:21.915817 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:10:21.916960 11140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: Bootstrap starting.
I20260806 01:10:21.917387 10892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: Bootstrap starting.
I20260806 01:10:21.920572 11140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:21.921468 10892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:21.921715 11140 log.cc:824] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:21.922782 10892 log.cc:824] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:21.926133 10892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: No bootstrap required, opened a new log
I20260806 01:10:21.941797 11140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: No bootstrap required, opened a new log
I20260806 01:10:21.944554 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.922239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.22.63.126:34597)
0806 01:10:21.922500 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.922508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.923921 (+ 1413us) server_negotiation.cc:407] Connection header received
0806 01:10:21.925603 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.925610 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.925694 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.925829 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.928343 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.929556 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.931691 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.932424 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.932561 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.937056 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.937063 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.937067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.937114 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.940862 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.942050 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.942056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.942060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.942230 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.942742 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.942748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.942751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.943121 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.943400 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.943957 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.944190 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:10:21.945060 10892 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:21.945499 10892 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:21.945619 10892 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3df9237b1b1412bb9a2ecc2b3b03a27, State: Initialized, Role: FOLLOWER
I20260806 01:10:21.946290 10892 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:21.950307 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:21.950639 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:21.952950 10892 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:21.953290 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.922921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.22.63.125:33403)
0806 01:10:21.923122 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.923129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.924766 (+ 1637us) server_negotiation.cc:407] Connection header received
0806 01:10:21.925978 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.925987 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.926094 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.926181 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.932104 (+ 5923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.933589 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.938820 (+ 5231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.939542 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.939643 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.943465 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.943472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.943477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.943520 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.947153 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.948251 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.948259 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.948262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.948675 (+ 413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.951316 (+ 2641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.951321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.951326 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.951718 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.951989 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.952589 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.952857 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:10:21.956065 11140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:21.956581 11140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:21.956729 11140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64d349195eeb48a882739d8eac47fea2, State: Initialized, Role: FOLLOWER
I20260806 01:10:21.957378 11140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:21.960439 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:21.925015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58176 (local address 127.22.63.124:43659)
0806 01:10:21.925213 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:21.925218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:21.937414 (+ 12196us) server_negotiation.cc:407] Connection header received
0806 01:10:21.937694 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:21.937703 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:21.937819 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:21.937897 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:21.939875 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.941012 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.946961 (+ 5949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:21.947575 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:21.947656 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:21.951076 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:21.951083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:21.951088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:21.951125 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:21.956575 (+ 5450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.957551 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.957559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.957565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.957744 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:21.958277 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:21.958285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:21.958291 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:21.958744 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:21.959080 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:21.959596 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 01:10:21.959924 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:10:21.966048 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:21.966583 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:21.967451 11140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:10:21.971047 11200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:10:21.971187 11200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 01:10:21.977469 10823 tablet.cc:2369] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:21.970373 10786 master_runner.cc:186] Error getting master registration for 127.22.63.125:33403: OK, instance_id { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" instance_seqno: 1785978621281721 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:10:21.985203 10786 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 01:10:21.990636 11219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:10:21.990862 11219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:10:21.990428 11021 master_runner.cc:186] Error getting master registration for 127.22.63.125:33403: OK, instance_id { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" instance_seqno: 1785978621281721 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:10:22.000041 11021 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:10:22.008960 11013 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } attempt: 5
Aug 06 01:10:22 dist-test-slave-7f01 krb5kdc[10769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:10:22.025599 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:22.009732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:43659 (local address 127.0.0.1:58192)
0806 01:10:22.010284 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:22.010307 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:22.010429 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:22.011546 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:22.011551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:22.012195 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:22.012417 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:22.012428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:22.013508 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:22.013516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:22.014409 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:22.014420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:22.015176 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:22.015182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:22.015435 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:22.016143 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:22.016167 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:22.018439 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:22.022802 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:22.022807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:22.022813 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:22.023352 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:22.023873 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:22.023877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:22.023879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:22.024004 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:22.024566 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:22.024574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:22.024733 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:10:22.024889 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 01:10:22.025188 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418}
I20260806 01:10:22.025627 11151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:22.010915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.22.63.124:43659)
0806 01:10:22.011090 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:22.011095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:22.011114 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:10:22.011162 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:22.011166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:22.011241 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:22.011334 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:22.012575 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:22.013356 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:22.014559 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:22.014985 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:22.015150 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:22.018639 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:22.018644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:22.018647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:22.018680 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:22.022649 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:22.023502 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:22.023507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:22.023510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:22.023707 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:22.024128 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:22.024133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:22.024137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:22.024434 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:22.024632 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:22.025018 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 01:10:22.025225 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:10:22.030578 11013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: Bootstrap starting.
I20260806 01:10:22.033812 11013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:22.034927 11013 log.cc:824] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:22.039649 11013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: No bootstrap required, opened a new log
I20260806 01:10:22.047278 11013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:22.047660 11013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:22.047821 11013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae3813aa20845d1a38e8d8dfbc6fb74, State: Initialized, Role: FOLLOWER
I20260806 01:10:22.048486 11013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:22.049474 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.049901 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:22.050714 11013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 01:10:22.066134 11061 tablet.cc:2369] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:22.074149 10904 master_runner.cc:263] No leader master found from master 2ae3813aa20845d1a38e8d8dfbc6fb74
I20260806 01:10:22.074321 10904 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 01:10:22.079319 11242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:10:22.079579 11242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:10:22.082942 11190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:10:22.083266 11190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:22.085088 11190 leader_election.cc:288] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3df9237b1b1412bb9a2ecc2b3b03a27 (127.22.63.126:34597), 2ae3813aa20845d1a38e8d8dfbc6fb74 (127.22.63.125:33403)
I20260806 01:10:22.086699 10853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64d349195eeb48a882739d8eac47fea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" is_pre_election: true
I20260806 01:10:22.087395 10853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d349195eeb48a882739d8eac47fea2 in term 0.
I20260806 01:10:22.088202 10969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64d349195eeb48a882739d8eac47fea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" is_pre_election: true
I20260806 01:10:22.088798 10969 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64d349195eeb48a882739d8eac47fea2 in term 0.
I20260806 01:10:22.092916 11056 leader_election.cc:302] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ae3813aa20845d1a38e8d8dfbc6fb74, 64d349195eeb48a882739d8eac47fea2; no voters:
I20260806 01:10:22.095279 11190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:10:22.095453 11190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:10:22.095556 11190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:22.099102 11190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:22.100518 10853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64d349195eeb48a882739d8eac47fea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27"
I20260806 01:10:22.100796 10853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:22.104470 10853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d349195eeb48a882739d8eac47fea2 in term 1.
I20260806 01:10:22.105091 11053 leader_election.cc:302] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64d349195eeb48a882739d8eac47fea2, f3df9237b1b1412bb9a2ecc2b3b03a27; no voters:
I20260806 01:10:22.113363 10969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64d349195eeb48a882739d8eac47fea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74"
I20260806 01:10:22.113612 10969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:22.117401 10969 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64d349195eeb48a882739d8eac47fea2 in term 1.
I20260806 01:10:22.118328 11190 leader_election.cc:288] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [CANDIDATE]: Term 1 election: Requested vote from peers f3df9237b1b1412bb9a2ecc2b3b03a27 (127.22.63.126:34597), 2ae3813aa20845d1a38e8d8dfbc6fb74 (127.22.63.125:33403)
I20260806 01:10:22.118973 11190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:22.119439 11190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [term 1 LEADER]: Becoming Leader. State: Replica: 64d349195eeb48a882739d8eac47fea2, State: Running, Role: LEADER
I20260806 01:10:22.120389 11190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } }
I20260806 01:10:22.139261 11245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d349195eeb48a882739d8eac47fea2. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.139896 11245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:22.146961 11250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:22.149531 11250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:22.171276 10969 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [term 1 FOLLOWER]: Refusing update from remote peer 64d349195eeb48a882739d8eac47fea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:22.172513 11245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:22.184283 10853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [term 1 FOLLOWER]: Refusing update from remote peer 64d349195eeb48a882739d8eac47fea2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:22.185506 11245 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:22.222447 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d349195eeb48a882739d8eac47fea2. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.222841 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:22.226897 11252 mvcc.cc:205] Tried to move back new op lower bound from 7315368436400144384 to 7315368436217561088. Current Snapshot: MvccSnapshot[applied={T|T < 7315368436400144384}]
I20260806 01:10:22.229440 11259 mvcc.cc:205] Tried to move back new op lower bound from 7315368436400144384 to 7315368436217561088. Current Snapshot: MvccSnapshot[applied={T|T < 7315368436400144384}]
I20260806 01:10:22.236663 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64d349195eeb48a882739d8eac47fea2. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.237021 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:22.249259 11257 mvcc.cc:205] Tried to move back new op lower bound from 7315368436400144384 to 7315368436217561088. Current Snapshot: MvccSnapshot[applied={T|T < 7315368436400144384}]
I20260806 01:10:22.256011 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.256291 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:22.257216 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.257540 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:22.261811 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.262127 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:22.269040 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64d349195eeb48a882739d8eac47fea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3df9237b1b1412bb9a2ecc2b3b03a27" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34597 } } peers { permanent_uuid: "2ae3813aa20845d1a38e8d8dfbc6fb74" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 33403 } } peers { permanent_uuid: "64d349195eeb48a882739d8eac47fea2" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 43659 } } }
I20260806 01:10:22.269311 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:10:22.269759 11250 catalog_manager.cc:1347] Generated new cluster ID: 4268a6e2c5ff44a19a2ed011f15ce8ba
I20260806 01:10:22.269868 11250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:22.295842 11250 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:10:22.300350 11250 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:22.334340 11250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64d349195eeb48a882739d8eac47fea2: Generated new TSK 0
I20260806 01:10:22.335534 11250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:10:22.973366 11200 catalog_manager.cc:1259] Loaded cluster ID: 4268a6e2c5ff44a19a2ed011f15ce8ba
I20260806 01:10:22.973524 11200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: loading cluster ID for follower catalog manager: success
I20260806 01:10:22.976285 11200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: acquiring CA information for follower catalog manager: success
I20260806 01:10:22.978123 11200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3df9237b1b1412bb9a2ecc2b3b03a27: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:10:23.081749 11242 catalog_manager.cc:1259] Loaded cluster ID: 4268a6e2c5ff44a19a2ed011f15ce8ba
I20260806 01:10:23.081875 11242 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: loading cluster ID for follower catalog manager: success
I20260806 01:10:23.084403 11242 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: acquiring CA information for follower catalog manager: success
I20260806 01:10:23.086254 11242 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2ae3813aa20845d1a38e8d8dfbc6fb74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:10:23.474289 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 10786
I20260806 01:10:23.475539 10887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:23.587067 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 10786
I20260806 01:10:23.618249 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 10904
I20260806 01:10:23.619498 11003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:23.731995 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 10904
I20260806 01:10:23.760957 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 11021
I20260806 01:10:23.762081 11134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:23.875716 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 11021
2026-08-06T01:10:23Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5264 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:10:24 dist-test-slave-7f01 krb5kdc[11286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:10:24 dist-test-slave-7f01 krb5kdc[11286](info): set up 2 sockets
Aug 06 01:10:24 dist-test-slave-7f01 krb5kdc[11286](info): commencing operation
krb5kdc: starting...
W20260806 01:10:25.217350 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:10:25 dist-test-slave-7f01 krb5kdc[11286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:10:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:10:25.519580 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37031
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:36303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:25.868340 11305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:25.868640 11305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:25.868794 11305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:25.876252 11305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:25.876405 11305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:25.876485 11305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:25.876585 11305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:25.876662 11305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:25.894389 11305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:25.896245 11305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:25.898249 11305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:25.917963 11316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:25.917870 11314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:25.918037 11320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:25.919276 11305 server_base.cc:1034] running on GCE node
I20260806 01:10:25.920598 11305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:25.923427 11305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:25.924785 11305 hybrid_clock.cc:648] HybridClock initialized: now 1785978625924688 us; error 77 us; skew 500 ppm
I20260806 01:10:25.925331 11305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:25 dist-test-slave-7f01 krb5kdc[11286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978625, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:25.929176 11305 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:25.930758 11305 webserver.cc:460] Webserver started at http://127.22.63.126:38249/ using document root <none> and password file <none>
I20260806 01:10:25.931497 11305 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:25.931655 11305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:25.932016 11305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:25.935159 11305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "e0fbfdf17d7747c4ae052c55cd723e86"
format_stamp: "Formatted at 2026-08-06 01:10:25 on dist-test-slave-7f01"
server_key: "fb3d2ade44025a094d6eb7f1a692ea15"
server_key_iv: "ddfabc534e56be0fca2f07e1f9eebf39"
server_key_version: "encryptionkey@0"
I20260806 01:10:25.936100 11305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "e0fbfdf17d7747c4ae052c55cd723e86"
format_stamp: "Formatted at 2026-08-06 01:10:25 on dist-test-slave-7f01"
server_key: "fb3d2ade44025a094d6eb7f1a692ea15"
server_key_iv: "ddfabc534e56be0fca2f07e1f9eebf39"
server_key_version: "encryptionkey@0"
I20260806 01:10:25.941586 11305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:10:25.945336 11332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:25.946646 11305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:25.946856 11305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "e0fbfdf17d7747c4ae052c55cd723e86"
format_stamp: "Formatted at 2026-08-06 01:10:25 on dist-test-slave-7f01"
server_key: "fb3d2ade44025a094d6eb7f1a692ea15"
server_key_iv: "ddfabc534e56be0fca2f07e1f9eebf39"
server_key_version: "encryptionkey@0"
I20260806 01:10:25.947072 11305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:25.978854 11305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:25.983335 11305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:25.983641 11305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:26.003239 11305 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37031
I20260806 01:10:26.003270 11418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37031 every 8 connection(s)
I20260806 01:10:26.004725 11305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:26.008728 11419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:26.010574 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 11305
I20260806 01:10:26.010825 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:10:26.011557 22781 external_mini_cluster.cc:1428] Setting key d11700f46e28702367449ddb8cb8c03f
I20260806 01:10:26.021376 11419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86: Bootstrap starting.
I20260806 01:10:26.025492 11419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:26.026924 11419 log.cc:824] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:10:26 dist-test-slave-7f01 krb5kdc[11286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:26.030092 11419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86: No bootstrap required, opened a new log
I20260806 01:10:26.036808 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:26.013694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.22.63.126:37031)
0806 01:10:26.014596 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:26.014608 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:26.014651 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:10:26.016344 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:26.016366 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:26.016745 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:26.017062 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:26.019103 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:26.020559 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:26.022397 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:26.023072 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:26.023202 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:26.029021 (+ 5819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:26.029029 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:26.029045 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:26.029080 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:26.032381 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:26.033220 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:26.033234 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:26.033244 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:26.033406 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:26.033824 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:26.033829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:26.033832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:26.034386 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:26.034769 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:26.035236 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 01:10:26.035699 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":416,"threads_started":1}
I20260806 01:10:26.038168 11419 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } }
I20260806 01:10:26.038579 11419 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:26.038707 11419 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fbfdf17d7747c4ae052c55cd723e86, State: Initialized, Role: FOLLOWER
I20260806 01:10:26.039366 11419 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } }
I20260806 01:10:26.039667 11419 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:26.039858 11419 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:26.040079 11419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:26.041025 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:10:26.042512 11419 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } }
I20260806 01:10:26.043027 11419 leader_election.cc:302] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0fbfdf17d7747c4ae052c55cd723e86; no voters:
I20260806 01:10:26.043706 11419 leader_election.cc:288] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:10:26.043859 11428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:26.044678 11428 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [term 1 LEADER]: Becoming Leader. State: Replica: e0fbfdf17d7747c4ae052c55cd723e86, State: Running, Role: LEADER
I20260806 01:10:26.045235 11428 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } }
I20260806 01:10:26.046125 11419 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:26.048527 11430 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0fbfdf17d7747c4ae052c55cd723e86. Latest consensus state: current_term: 1 leader_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } } }
I20260806 01:10:26.048586 11429 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fbfdf17d7747c4ae052c55cd723e86" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37031 } } }
I20260806 01:10:26.048960 11429 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:26.048959 11430 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:26.050922 11438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:26.056445 11438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:26.066377 11438 catalog_manager.cc:1347] Generated new cluster ID: 730befffe6c047f4997379ab479baffa
I20260806 01:10:26.066500 11438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:26.087275 11438 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:10:26.088698 11438 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:26.096369 11438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0fbfdf17d7747c4ae052c55cd723e86: Generated new TSK 0
I20260806 01:10:26.097391 11438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:10:26.154359 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 11305
I20260806 01:10:26.155659 11412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:26.254294 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 11305
2026-08-06T01:10:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2372 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:10:26 dist-test-slave-7f01 krb5kdc[11463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:10:26 dist-test-slave-7f01 krb5kdc[11463](info): set up 2 sockets
Aug 06 01:10:26 dist-test-slave-7f01 krb5kdc[11463](info): commencing operation
krb5kdc: starting...
W20260806 01:10:27.570077 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.008s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:10:27 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:10:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:10:27.885205 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:28.238987 11486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:28.239292 11486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:28.239460 11486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:28.246876 11486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:28.246994 11486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:28.247093 11486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:28.247192 11486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:28.247278 11486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:28.257447 11486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:28.258839 11486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:28.260185 11486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:28.272536 11498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:28.272447 11499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:28.272483 11503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:28.273641 11486 server_base.cc:1034] running on GCE node
I20260806 01:10:28.274513 11486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:28.276021 11486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:28.277235 11486 hybrid_clock.cc:648] HybridClock initialized: now 1785978628277194 us; error 35 us; skew 500 ppm
I20260806 01:10:28.277606 11486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:28 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:28.280998 11486 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:28.282495 11486 webserver.cc:460] Webserver started at http://127.22.63.126:44921/ using document root <none> and password file <none>
I20260806 01:10:28.283161 11486 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:28.283262 11486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:28.283516 11486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:28.286578 11486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "d190f8b3787f411291d7c95f910cb110"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "8204dac3e31657c0a776b5d850238d25"
server_key_iv: "aa7c11a1ec259173b344d2019fb470a8"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.287344 11486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "d190f8b3787f411291d7c95f910cb110"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "8204dac3e31657c0a776b5d850238d25"
server_key_iv: "aa7c11a1ec259173b344d2019fb470a8"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.292632 11486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:10:28.296067 11515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:28.297421 11486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:10:28.297611 11486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "d190f8b3787f411291d7c95f910cb110"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "8204dac3e31657c0a776b5d850238d25"
server_key_iv: "aa7c11a1ec259173b344d2019fb470a8"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.297809 11486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:28.314908 11486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:28.319270 11486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:28.319574 11486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:28.338842 11486 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:43753
I20260806 01:10:28.338882 11599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:43753 every 8 connection(s)
I20260806 01:10:28.340279 11486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:28.344084 11600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:28.348153 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 11486
I20260806 01:10:28.348392 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:10:28.349267 22781 external_mini_cluster.cc:1428] Setting key a82ef0e9c93c7dea8d5c9ff27a09a70f
I20260806 01:10:28.357781 11600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap starting.
I20260806 01:10:28.361012 11600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:28.362039 11600 log.cc:824] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Log is configured to *not* fsync() on all Append() calls
Aug 06 01:10:28 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:28.365039 11600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: No bootstrap required, opened a new log
I20260806 01:10:28.373545 11600 raft_consensus.cc:348] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:28.373967 11600 raft_consensus.cc:374] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:28.374095 11600 raft_consensus.cc:729] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Initialized, Role: FOLLOWER
I20260806 01:10:28.374588 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:28.351555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.22.63.126:43753)
0806 01:10:28.352449 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:28.352462 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:28.352506 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:10:28.354225 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:28.354272 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:28.354721 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:28.355188 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:28.356907 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:28.358443 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:28.360056 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:28.360768 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:28.360991 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:28.365686 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:28.365695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:28.365710 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:28.365755 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:28.369602 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:28.370427 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:28.370435 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:28.370442 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:28.370640 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:28.371225 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:28.371231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:28.371235 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:28.371997 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:28.372268 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:28.372770 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 01:10:28.373252 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":423,"threads_started":1}
I20260806 01:10:28.374706 11600 consensus_queue.cc:260] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:28.375002 11600 raft_consensus.cc:388] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:28.375167 11600 raft_consensus.cc:482] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:28.375375 11600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:28.377813 11600 raft_consensus.cc:504] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:28.378321 11600 leader_election.cc:302] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d190f8b3787f411291d7c95f910cb110; no voters:
I20260806 01:10:28.378924 11600 leader_election.cc:288] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:10:28.379098 11605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:28.381222 11600 sys_catalog.cc:564] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:28.385108 11605 raft_consensus.cc:686] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 LEADER]: Becoming Leader. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Running, Role: LEADER
I20260806 01:10:28.385917 11605 consensus_queue.cc:237] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:28.389468 11608 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d190f8b3787f411291d7c95f910cb110. Latest consensus state: current_term: 1 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:28.389885 11608 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:28.389652 11606 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:28.390069 11606 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:28.391984 11628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:28.396322 11628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:28.404938 11628 catalog_manager.cc:1347] Generated new cluster ID: f0b1a5af6a844bdb88477a23bbff2b88
I20260806 01:10:28.405082 11628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:28.442512 11628 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:10:28.444039 11628 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:28.450497 11628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Generated new TSK 0
I20260806 01:10:28.451362 11628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:10:28.523203 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:28.877530 11636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:28.877823 11636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:28.877988 11636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:28.885947 11636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:28.886068 11636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:28.886270 11636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:28.896520 11636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:28.897950 11636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:28.899322 11636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:28.913708 11643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:28.913755 11644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:28.913794 11652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:28.914794 11636 server_base.cc:1034] running on GCE node
I20260806 01:10:28.915452 11636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:28.916572 11636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:28.917796 11636 hybrid_clock.cc:648] HybridClock initialized: now 1785978628917753 us; error 28 us; skew 500 ppm
I20260806 01:10:28.918109 11636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:28 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:28.921554 11636 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:28.923081 11636 webserver.cc:460] Webserver started at http://127.22.63.65:41331/ using document root <none> and password file <none>
I20260806 01:10:28.923835 11636 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:28.923944 11636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:28.924206 11636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:28.927245 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "d167b7a4e4294d679b5e0d970d75ce39"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "cc63e003c9836dcab39dc8b5be25985b"
server_key_iv: "77fc0376f0be552b8c157845128e0296"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.928105 11636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "d167b7a4e4294d679b5e0d970d75ce39"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "cc63e003c9836dcab39dc8b5be25985b"
server_key_iv: "77fc0376f0be552b8c157845128e0296"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.933642 11636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:10:28.937242 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:28.938730 11636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:28.938923 11636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "d167b7a4e4294d679b5e0d970d75ce39"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "cc63e003c9836dcab39dc8b5be25985b"
server_key_iv: "77fc0376f0be552b8c157845128e0296"
server_key_version: "encryptionkey@0"
I20260806 01:10:28.939143 11636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:28.957348 11636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:28.961983 11636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:28.962313 11636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:28.963402 11636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:28.964898 11636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:28.965020 11636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:28.965181 11636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:28.965256 11636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:29.000325 11636 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44967
I20260806 01:10:29.000370 11855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44967 every 8 connection(s)
I20260806 01:10:29.001766 11636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:29.004774 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 11636
I20260806 01:10:29.005034 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:10:29.005963 22781 external_mini_cluster.cc:1428] Setting key e649ca29e3a947e099b7e29f940fb271
Aug 06 01:10:29 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978628, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:29.027014 11863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:29.004606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:43313 (local address 127.22.63.126:43753)
0806 01:10:29.005915 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:29.005922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:29.007311 (+ 1389us) server_negotiation.cc:407] Connection header received
0806 01:10:29.009885 (+ 2574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:29.009890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:29.009961 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:29.010055 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:29.012322 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.013123 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.015655 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.016135 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.016241 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:29.020545 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:29.020550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:29.020553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:29.020588 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:29.023802 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:29.024573 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:29.024579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:29.024583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:29.024723 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:29.025164 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:29.025169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:29.025178 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:29.025447 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:29.025649 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:29.026356 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 01:10:29.026587 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1163,"thread_start_us":1020,"threads_started":1}
I20260806 01:10:29.028080 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:29.005351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:43313)
0806 01:10:29.007156 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:29.007196 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:29.009562 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:29.010366 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:29.010381 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:29.011231 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:29.012109 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:29.012137 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.013303 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.013311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:29.015485 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:29.015503 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.016391 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.016400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:29.016731 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:29.017484 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:29.017527 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:29.020307 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:29.023950 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:29.023960 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:29.023977 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:29.024437 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:29.024839 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:29.024844 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:29.024848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:29.025036 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:29.025607 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:29.025615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:29.025983 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 01:10:29.026260 (+ 277us) client_negotiation.cc:240] Negotiation successful
0806 01:10:29.026835 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1465,"thread_start_us":1153,"threads_started":1}
I20260806 01:10:29.029767 11860 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:29.030104 11860 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:29.030870 11860 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:29.033385 11537 ts_manager.cc:194] Registered new tserver with Master: d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65:44967)
I20260806 01:10:29.035670 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:43313
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:10:29.077430 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:29.426230 11868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:29.426601 11868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:29.426703 11868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:29.434266 11868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:29.434392 11868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:29.434508 11868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:29.444561 11868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:29.445837 11868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:29.447271 11868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:29.461078 11878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:29.461098 11875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:29.461928 11880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:29.461987 11868 server_base.cc:1034] running on GCE node
I20260806 01:10:29.462764 11868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:29.463851 11868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:29.465083 11868 hybrid_clock.cc:648] HybridClock initialized: now 1785978629465033 us; error 58 us; skew 500 ppm
I20260806 01:10:29.465415 11868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:29 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978629, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:29.468734 11868 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:29.470402 11868 webserver.cc:460] Webserver started at http://127.22.63.66:36771/ using document root <none> and password file <none>
I20260806 01:10:29.471107 11868 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:29.471227 11868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:29.471485 11868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:29.474633 11868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "4256d56e9bb2407b9c41380efe3c3a44"
format_stamp: "Formatted at 2026-08-06 01:10:29 on dist-test-slave-7f01"
server_key: "db43df2679343b0a7c5ed065d0178e15"
server_key_iv: "a8f615c5a3cd1d3ba26188ad586d2bac"
server_key_version: "encryptionkey@0"
I20260806 01:10:29.475386 11868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "4256d56e9bb2407b9c41380efe3c3a44"
format_stamp: "Formatted at 2026-08-06 01:10:29 on dist-test-slave-7f01"
server_key: "db43df2679343b0a7c5ed065d0178e15"
server_key_iv: "a8f615c5a3cd1d3ba26188ad586d2bac"
server_key_version: "encryptionkey@0"
I20260806 01:10:29.480563 11868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 01:10:29.484532 11899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:29.485987 11868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:29.486163 11868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "4256d56e9bb2407b9c41380efe3c3a44"
format_stamp: "Formatted at 2026-08-06 01:10:29 on dist-test-slave-7f01"
server_key: "db43df2679343b0a7c5ed065d0178e15"
server_key_iv: "a8f615c5a3cd1d3ba26188ad586d2bac"
server_key_version: "encryptionkey@0"
I20260806 01:10:29.486351 11868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:29.508678 11868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:29.513075 11868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:29.513383 11868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:29.514397 11868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:29.515894 11868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:29.515976 11868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:29.516108 11868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:29.516161 11868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:29.552613 11868 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:33541
I20260806 01:10:29.552683 12081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:33541 every 8 connection(s)
I20260806 01:10:29.553969 11868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:29.559149 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 11868
I20260806 01:10:29.559355 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:10:29.559859 22781 external_mini_cluster.cc:1428] Setting key f169f50c531e11205674fa4ffa3da43f
Aug 06 01:10:29 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978629, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:29.575105 12086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:29.556744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:48289 (local address 127.22.63.126:43753)
0806 01:10:29.557219 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:29.557224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:29.558096 (+ 872us) server_negotiation.cc:407] Connection header received
0806 01:10:29.560176 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:29.560182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:29.560287 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:29.560411 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:29.561929 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.562642 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.564297 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.564736 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.564884 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:29.568728 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:29.568733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:29.568736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:29.568770 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:29.571645 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:29.572448 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:29.572458 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:29.572461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:29.572590 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:29.573054 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:29.573058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:29.573060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:29.573434 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:29.573607 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:29.574474 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 01:10:29.574711 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":258,"threads_started":1}
I20260806 01:10:29.576272 12087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:29.557170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:48289)
0806 01:10:29.557922 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:29.557968 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:29.559867 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:29.560646 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:29.560653 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:29.561179 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:29.561756 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:29.561773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.562804 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.562816 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:29.564143 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:29.564154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:29.564986 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:29.564993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:29.565233 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:29.565949 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:29.565979 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:29.568484 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:29.571814 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:29.571827 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:29.571844 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:29.572299 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:29.572711 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:29.572715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:29.572719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:29.572920 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:29.573575 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:29.573584 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:29.574030 (+ 446us) client_negotiation.cc:769] Sending connection context
0806 01:10:29.574354 (+ 324us) client_negotiation.cc:240] Negotiation successful
0806 01:10:29.574934 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":299,"threads_started":1}
I20260806 01:10:29.577945 12082 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:29.578305 12082 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:29.579332 12082 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:29.581187 11537 ts_manager.cc:194] Registered new tserver with Master: 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541)
I20260806 01:10:29.582634 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:48289
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:10:29.632050 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:29.989111 12092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:29.989408 12092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:29.989507 12092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:29.996919 12092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:29.997026 12092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:29.997138 12092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:30.007206 12092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:30.008471 12092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:30.009747 12092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:30.022794 12101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:30.022927 12106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:30.022980 12104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:30.024384 12092 server_base.cc:1034] running on GCE node
I20260806 01:10:30.025068 12092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:30.026038 12092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:30.027278 12092 hybrid_clock.cc:648] HybridClock initialized: now 1785978630027227 us; error 40 us; skew 500 ppm
I20260806 01:10:30.027612 12092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:30 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978630, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:30.030838 12092 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:30.032490 12092 webserver.cc:460] Webserver started at http://127.22.63.67:37053/ using document root <none> and password file <none>
I20260806 01:10:30.033210 12092 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:30.033360 12092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:30.033685 12092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:30.036763 12092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
format_stamp: "Formatted at 2026-08-06 01:10:30 on dist-test-slave-7f01"
server_key: "9a5d931d0fe449276e6609b3ee465b87"
server_key_iv: "781fe1d371efa062e86a7401f14bcb44"
server_key_version: "encryptionkey@0"
I20260806 01:10:30.037576 12092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
format_stamp: "Formatted at 2026-08-06 01:10:30 on dist-test-slave-7f01"
server_key: "9a5d931d0fe449276e6609b3ee465b87"
server_key_iv: "781fe1d371efa062e86a7401f14bcb44"
server_key_version: "encryptionkey@0"
I20260806 01:10:30.038884 11860 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:30.043102 12092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:10:30.046715 12118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:30.048151 12092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:10:30.048334 12092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
format_stamp: "Formatted at 2026-08-06 01:10:30 on dist-test-slave-7f01"
server_key: "9a5d931d0fe449276e6609b3ee465b87"
server_key_iv: "781fe1d371efa062e86a7401f14bcb44"
server_key_version: "encryptionkey@0"
I20260806 01:10:30.048542 12092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:30.110795 12092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:30.115103 12092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:30.115422 12092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:30.116525 12092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:30.117944 12092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:30.118052 12092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:30.118212 12092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:30.118289 12092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:30.153688 12092 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:33739
I20260806 01:10:30.153789 12301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:33739 every 8 connection(s)
I20260806 01:10:30.155061 12092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:30.164224 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 12092
I20260806 01:10:30.164444 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:10:30.165068 22781 external_mini_cluster.cc:1428] Setting key b077b93725ce630d444c2399c46c71ad
Aug 06 01:10:30 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978630, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:30.176396 12305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.157827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:45013 (local address 127.22.63.126:43753)
0806 01:10:30.158297 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.158303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.159349 (+ 1046us) server_negotiation.cc:407] Connection header received
0806 01:10:30.161191 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.161195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.161265 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.161363 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:30.162848 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.163642 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.165985 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.166555 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.166714 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.170408 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:30.170413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:30.170416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:30.170451 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:30.173172 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.173923 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.173933 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.173935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.174080 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.174598 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.174601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.174603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.174882 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:30.175025 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.175880 (+ 855us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.176070 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":214,"threads_started":1}
I20260806 01:10:30.177353 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.158468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:45013)
0806 01:10:30.159177 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.159223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.160931 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.161575 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.161582 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.162115 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:30.162690 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.162707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.163846 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.163854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.165824 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.165838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.166802 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.166808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.167059 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.167853 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:30.167882 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:30.170181 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:30.173319 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:30.173328 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:30.173345 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:30.173777 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:30.174201 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:30.174206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:30.174210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:30.174427 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:30.175022 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:30.175028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:30.175355 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.175683 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.176187 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":265,"threads_started":1}
I20260806 01:10:30.178968 12302 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:30.179358 12302 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:30.180241 12302 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:30.181898 11537 ts_manager.cc:194] Registered new tserver with Master: 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739)
I20260806 01:10:30.182803 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:45013
I20260806 01:10:30.193426 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:10:30.211621 12305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.198044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.22.63.126:43753)
0806 01:10:30.198249 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.198257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.198619 (+ 362us) server_negotiation.cc:407] Connection header received
0806 01:10:30.198842 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.198847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.198915 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.199019 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:30.200604 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.201442 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.202949 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.203358 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.203501 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.206090 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:30.206095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:30.206098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:30.206132 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:30.208884 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.209595 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.209607 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.209610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.209734 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.210134 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.210140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.210143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.210486 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:30.210755 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.211068 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.211295 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:10:30.215730 11537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:10:30.222643 11537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:10:30.245481 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.235898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48046 (local address 127.22.63.65:44967)
0806 01:10:30.236503 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.236512 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.236540 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:10:30.236638 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.236646 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.236893 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.237114 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:30.237956 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.239328 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.242258 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.243808 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.243940 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.244213 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.244943 (+ 730us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.245139 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":1091,"threads_started":1}
I20260806 01:10:30.246701 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.235186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44967 (local address 127.0.0.1:48046)
0806 01:10:30.236035 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.236065 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.236204 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.237152 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.237161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.237196 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:30.237683 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.237705 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.239457 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.239464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.242067 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.242086 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.244451 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.244458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.244663 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.244717 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.246089 (+ 1372us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.246313 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":891,"threads_started":1}
I20260806 01:10:30.249274 12326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.236865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33739 (local address 127.0.0.1:54590)
0806 01:10:30.240277 (+ 3412us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.240297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.240418 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.241370 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.241375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.241406 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:30.241889 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.241903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.244027 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.244033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.246221 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.246234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.248369 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.248376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.248559 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.248591 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.248699 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.248870 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3269,"thread_start_us":3042,"threads_started":1}
I20260806 01:10:30.249387 12327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.237278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.22.63.67:33739)
0806 01:10:30.238394 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.238406 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.240563 (+ 2157us) server_negotiation.cc:407] Connection header received
0806 01:10:30.240675 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.240682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.240907 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.241092 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:30.242126 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.243863 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.246404 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.247858 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.247998 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.248218 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.248809 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.249017 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":896,"thread_start_us":429,"threads_started":1}
I20260806 01:10:30.253099 11744 tablet_service.cc:1467] Processing CreateTablet for tablet 8242e43a304e4650a6e42cf478280a8b (DEFAULT_TABLE table=test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:30.253821 12195 tablet_service.cc:1467] Processing CreateTablet for tablet 8242e43a304e4650a6e42cf478280a8b (DEFAULT_TABLE table=test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:30.254704 11744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8242e43a304e4650a6e42cf478280a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:30.255142 12323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.240244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:33541 (local address 127.0.0.1:46608)
0806 01:10:30.247145 (+ 6901us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.247166 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.247271 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.249221 (+ 1950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.249227 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.249264 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:30.249451 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.249462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.251448 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.251453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.252992 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.253005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.254504 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.254519 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.254703 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.254742 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.254832 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.254945 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6795,"thread_start_us":8682,"threads_started":1}
I20260806 01:10:30.255313 12195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8242e43a304e4650a6e42cf478280a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:30.255841 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.240522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46608 (local address 127.22.63.66:33541)
0806 01:10:30.248255 (+ 7733us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.248264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.248292 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:10:30.248393 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.248401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.248645 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.249629 (+ 984us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:30.249862 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.251265 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.253171 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.254322 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.254474 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.254674 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.255389 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.255542 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7540,"thread_start_us":7439,"threads_started":1}
I20260806 01:10:30.262558 11972 tablet_service.cc:1467] Processing CreateTablet for tablet 8242e43a304e4650a6e42cf478280a8b (DEFAULT_TABLE table=test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:30.263705 11972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8242e43a304e4650a6e42cf478280a8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:30.270103 12333 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Bootstrap starting.
I20260806 01:10:30.270905 12336 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Bootstrap starting.
I20260806 01:10:30.274812 12336 tablet_bootstrap.cc:654] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:30.276211 12336 log.cc:824] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:30.277233 12338 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Bootstrap starting.
I20260806 01:10:30.280124 12338 tablet_bootstrap.cc:654] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:30.280269 12333 tablet_bootstrap.cc:654] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:30.281476 12333 log.cc:824] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:30.281458 12338 log.cc:824] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:30.282660 12336 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: No bootstrap required, opened a new log
I20260806 01:10:30.282951 12336 ts_tablet_manager.cc:1397] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260806 01:10:30.284890 12333 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: No bootstrap required, opened a new log
I20260806 01:10:30.285166 12333 ts_tablet_manager.cc:1397] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260806 01:10:30.291078 12336 raft_consensus.cc:348] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.291568 12336 raft_consensus.cc:374] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:30.291682 12336 raft_consensus.cc:729] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d167b7a4e4294d679b5e0d970d75ce39, State: Initialized, Role: FOLLOWER
I20260806 01:10:30.291983 12338 tablet_bootstrap.cc:492] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: No bootstrap required, opened a new log
I20260806 01:10:30.292336 12338 ts_tablet_manager.cc:1397] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260806 01:10:30.292395 12336 consensus_queue.cc:260] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.292702 12333 raft_consensus.cc:348] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.293228 12333 raft_consensus.cc:374] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:30.294256 12336 ts_tablet_manager.cc:1428] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260806 01:10:30.293380 12333 raft_consensus.cc:729] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca9c8dc55c4cbbb0ee4193917e823b, State: Initialized, Role: FOLLOWER
I20260806 01:10:30.295233 12333 consensus_queue.cc:260] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.296407 12302 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:30.296975 12333 ts_tablet_manager.cc:1428] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Time spent starting tablet: real 0.012s user 0.005s sys 0.005s
I20260806 01:10:30.302277 12338 raft_consensus.cc:348] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.302616 12338 raft_consensus.cc:374] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:30.302688 12338 raft_consensus.cc:729] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4256d56e9bb2407b9c41380efe3c3a44, State: Initialized, Role: FOLLOWER
I20260806 01:10:30.303217 12338 consensus_queue.cc:260] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.304306 12082 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:30.304780 12338 ts_tablet_manager.cc:1428] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Time spent starting tablet: real 0.012s user 0.000s sys 0.012s
W20260806 01:10:30.306052 12083 tablet.cc:2369] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:10:30.406988 12304 tablet.cc:2369] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:30.450230 12345 raft_consensus.cc:482] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:10:30.450611 12345 raft_consensus.cc:504] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.452739 12345 leader_election.cc:288] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739), 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541)
I20260806 01:10:30.459872 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.453022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33739 (local address 127.22.63.65:35213)
0806 01:10:30.453712 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.453728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.453947 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.454416 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.454419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.454444 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:30.454780 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.454788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.455809 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.455817 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.457804 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.457819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.459207 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.459211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.459328 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.459365 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.459441 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.459533 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":353,"threads_started":1}
I20260806 01:10:30.459985 12327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.453123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:35213 (local address 127.22.63.67:33739)
0806 01:10:30.453397 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.453406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.453879 (+ 473us) server_negotiation.cc:407] Connection header received
0806 01:10:30.454100 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.454107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.454203 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.454306 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:30.454936 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.455653 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.457990 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.459001 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.459181 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.459261 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.459505 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.459643 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:10:30.460872 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.454051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:58815 (local address 127.22.63.66:33541)
0806 01:10:30.454304 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.454310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.454667 (+ 357us) server_negotiation.cc:407] Connection header received
0806 01:10:30.454878 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.454885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.454951 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.455057 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:30.455580 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.456545 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.458604 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.459853 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.460232 (+ 379us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.460325 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.460551 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.460685 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 01:10:30.460938 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.453885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:33541 (local address 127.22.63.65:58815)
0806 01:10:30.454552 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:30.454574 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:30.454722 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:30.455076 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:30.455080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:30.455104 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:30.455432 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.455440 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.456710 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.456715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.458435 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:30.458446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.460097 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.460102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:30.460400 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.460423 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:10:30.460493 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 01:10:30.460572 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":319,"threads_started":1}
I20260806 01:10:30.461199 12230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8242e43a304e4650a6e42cf478280a8b" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" is_pre_election: true
I20260806 01:10:30.461872 12230 raft_consensus.cc:2454] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 0.
I20260806 01:10:30.461880 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8242e43a304e4650a6e42cf478280a8b" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4256d56e9bb2407b9c41380efe3c3a44" is_pre_election: true
I20260806 01:10:30.462417 12006 raft_consensus.cc:2454] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 0.
I20260806 01:10:30.462669 11667 leader_election.cc:302] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ca9c8dc55c4cbbb0ee4193917e823b, d167b7a4e4294d679b5e0d970d75ce39; no voters:
I20260806 01:10:30.463099 12345 raft_consensus.cc:2781] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:10:30.463274 12345 raft_consensus.cc:482] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:10:30.463414 12345 raft_consensus.cc:3037] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:30.466024 12345 raft_consensus.cc:504] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.466662 12345 leader_election.cc:288] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 election: Requested vote from peers 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739), 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541)
I20260806 01:10:30.467237 12230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8242e43a304e4650a6e42cf478280a8b" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
I20260806 01:10:30.467327 12006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8242e43a304e4650a6e42cf478280a8b" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4256d56e9bb2407b9c41380efe3c3a44"
I20260806 01:10:30.467463 12230 raft_consensus.cc:3037] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:30.467612 12006 raft_consensus.cc:3037] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:30.469833 12230 raft_consensus.cc:2454] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 1.
I20260806 01:10:30.469928 12006 raft_consensus.cc:2454] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 1.
I20260806 01:10:30.470357 11667 leader_election.cc:302] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53ca9c8dc55c4cbbb0ee4193917e823b, d167b7a4e4294d679b5e0d970d75ce39; no voters:
I20260806 01:10:30.470674 12345 raft_consensus.cc:2781] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:30.471297 12345 raft_consensus.cc:686] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 1 LEADER]: Becoming Leader. State: Replica: d167b7a4e4294d679b5e0d970d75ce39, State: Running, Role: LEADER
I20260806 01:10:30.471997 12345 consensus_queue.cc:237] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } }
I20260806 01:10:30.477377 11536 catalog_manager.cc:5495] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 reported cstate change: term changed from 0 to 1, leader changed from <none> to d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "d167b7a4e4294d679b5e0d970d75ce39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } health_report { overall_health: UNKNOWN } } }
W20260806 01:10:30.504506 11861 tablet.cc:2369] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:30.544198 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.537735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48052 (local address 127.22.63.65:44967)
0806 01:10:30.537938 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.537943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.537962 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:10:30.538196 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.538201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.538317 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.538453 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:10:30.539108 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.539981 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.541624 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.542163 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.542270 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.542856 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:10:30.543524 (+ 668us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:10:30.543660 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.543826 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.543920 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:10:30.553088 12006 raft_consensus.cc:1261] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Refusing update from remote peer d167b7a4e4294d679b5e0d970d75ce39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:30.553284 12230 raft_consensus.cc:1261] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Refusing update from remote peer d167b7a4e4294d679b5e0d970d75ce39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:30.554368 12352 consensus_queue.cc:1035] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:30.554872 12345 consensus_queue.cc:1035] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:30.574536 12365 mvcc.cc:205] Tried to move back new op lower bound from 7315368470734536704 to 7315368470418272256. Current Snapshot: MvccSnapshot[applied={T|T < 7315368470734536704}]
I20260806 01:10:30.587540 11536 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:30.587875 11536 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:30.592902 11536 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Sending DeleteTablet for 3 replicas of tablet 8242e43a304e4650a6e42cf478280a8b
I20260806 01:10:30.594091 11744 tablet_service.cc:1514] Processing DeleteTablet for tablet 8242e43a304e4650a6e42cf478280a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:48046
I20260806 01:10:30.594316 12195 tablet_service.cc:1514] Processing DeleteTablet for tablet 8242e43a304e4650a6e42cf478280a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:54590
I20260806 01:10:30.594739 11972 tablet_service.cc:1514] Processing DeleteTablet for tablet 8242e43a304e4650a6e42cf478280a8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:30 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46608
I20260806 01:10:30.595112 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 11636
I20260806 01:10:30.607214 12375 tablet_replica.cc:330] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: stopping tablet replica
I20260806 01:10:30.607829 12375 raft_consensus.cc:2229] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:30.608474 12375 raft_consensus.cc:2258] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:30.610891 12374 tablet_replica.cc:330] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: stopping tablet replica
I20260806 01:10:30.611141 12373 tablet_replica.cc:330] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: stopping tablet replica
I20260806 01:10:30.611200 12375 ts_tablet_manager.cc:1905] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:30.611387 12374 raft_consensus.cc:2229] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:30.611703 12373 raft_consensus.cc:2229] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:10:30.612604 12373 raft_consensus.cc:2258] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:30.614900 12373 ts_tablet_manager.cc:1905] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:30.615592 12374 raft_consensus.cc:2258] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:30.619554 12374 ts_tablet_manager.cc:1905] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:30.620491 12375 ts_tablet_manager.cc:1918] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:30.620769 12375 log.cc:1196] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/8242e43a304e4650a6e42cf478280a8b
I20260806 01:10:30.621469 12375 ts_tablet_manager.cc:1939] T 8242e43a304e4650a6e42cf478280a8b P 4256d56e9bb2407b9c41380efe3c3a44: Deleting consensus metadata
I20260806 01:10:30.622898 12373 ts_tablet_manager.cc:1918] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:30.623162 12373 log.cc:1196] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/8242e43a304e4650a6e42cf478280a8b
I20260806 01:10:30.623782 11516 catalog_manager.cc:4841] TS 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541): tablet 8242e43a304e4650a6e42cf478280a8b (table test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]) successfully deleted
I20260806 01:10:30.623903 12373 ts_tablet_manager.cc:1939] T 8242e43a304e4650a6e42cf478280a8b P d167b7a4e4294d679b5e0d970d75ce39: Deleting consensus metadata
Aug 06 01:10:30 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:10:30.625511 11516 catalog_manager.cc:4841] TS d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65:44967): tablet 8242e43a304e4650a6e42cf478280a8b (table test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]) successfully deleted
I20260806 01:10:30.631711 12374 ts_tablet_manager.cc:1918] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:30.631987 12374 log.cc:1196] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/8242e43a304e4650a6e42cf478280a8b
I20260806 01:10:30.632630 12374 ts_tablet_manager.cc:1939] T 8242e43a304e4650a6e42cf478280a8b P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting consensus metadata
I20260806 01:10:30.634883 11516 catalog_manager.cc:4841] TS 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739): tablet 8242e43a304e4650a6e42cf478280a8b (table test-table [id=338e5256ae0a4b9caf59e5d0b464dc7b]) successfully deleted
I20260806 01:10:30.635685 12324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.596757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.22.63.65:44967)
0806 01:10:30.597002 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.597011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.604421 (+ 7410us) server_negotiation.cc:407] Connection header received
0806 01:10:30.605232 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.605243 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.605353 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.605459 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:30.611718 (+ 6259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.613079 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.616145 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.616882 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.616993 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.628967 (+ 11974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:30.628979 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:30.628994 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:30.629039 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:30.632451 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.633353 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.633362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.633364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.633529 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.633987 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.633991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.633993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.634353 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:30.634906 (+ 553us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.635075 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.635337 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:10:30.636574 11849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:30.737306 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 11636
I20260806 01:10:30.763417 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 11868
Aug 06 01:10:30 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:10:30.785408 12332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.764335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46610 (local address 127.22.63.66:33541)
0806 01:10:30.764578 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.764587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.764613 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:10:30.764890 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.764896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.764980 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.765118 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:30.766581 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.767453 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.769105 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.769560 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.769714 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.779176 (+ 9462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:30.779196 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:30.779212 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:30.779256 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:30.782324 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.783087 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.783095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.783097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.783224 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.783637 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.783648 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.783651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.784126 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:30.784491 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.784795 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.785060 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 01:10:30.786214 12071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:30.878685 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 11868
I20260806 01:10:30.905035 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 12092
Aug 06 01:10:30 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:10:30.926400 12327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:30.905911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.22.63.67:33739)
0806 01:10:30.906135 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:30.906141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:30.906164 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:10:30.906337 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:30.906350 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:30.906420 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:30.906530 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:30.907831 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.908582 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.909854 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:30.910247 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:30.910390 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:30.921119 (+ 10729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:30.921139 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:30.921157 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:30.921197 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:30.923905 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.924566 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.924573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.924576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.924703 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:30.925068 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:30.925071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:30.925072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:30.925441 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:30.925669 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:30.925942 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 01:10:30.926119 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:10:30.927228 12296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:31.017309 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 12092
I20260806 01:10:31.044960 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 11486
I20260806 01:10:31.046191 11589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:31.157352 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 11486
I20260806 01:10:31.189464 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--webserver_interface=127.22.63.126
--webserver_port=44921
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:31.538141 12384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:31.538440 12384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:31.538533 12384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:31.545691 12384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:31.545806 12384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:31.545862 12384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:31.545919 12384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:31.545969 12384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:31.557145 12384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:31.558451 12384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:31.559792 12384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:31.585758 12398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:31.585834 12396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:31.585758 12395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:31.586678 12384 server_base.cc:1034] running on GCE node
I20260806 01:10:31.587603 12384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:31.589275 12384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:31.590546 12384 hybrid_clock.cc:648] HybridClock initialized: now 1785978631590496 us; error 61 us; skew 500 ppm
I20260806 01:10:31.591048 12384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:31 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978631, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:31.594810 12384 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:31.596688 12384 webserver.cc:460] Webserver started at http://127.22.63.126:44921/ using document root <none> and password file <none>
I20260806 01:10:31.597481 12384 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:31.597659 12384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:31.603713 12384 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:10:31.606760 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:31.608392 12384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:10:31.608588 12384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "d190f8b3787f411291d7c95f910cb110"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "8204dac3e31657c0a776b5d850238d25"
server_key_iv: "aa7c11a1ec259173b344d2019fb470a8"
server_key_version: "encryptionkey@0"
I20260806 01:10:31.609700 12384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:31.630643 12384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:31.635089 12384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:31.635396 12384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:31.655229 12384 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:43753
I20260806 01:10:31.655256 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:43753 every 8 connection(s)
I20260806 01:10:31.656857 12384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:31.660331 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 12384
I20260806 01:10:31.661396 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:44967
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=41331
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:31.666060 12502 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:10:31.671515 12502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap starting.
I20260806 01:10:31.693193 12502 log.cc:824] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:31.706241 12502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:10:31.706825 12502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap complete.
I20260806 01:10:31.716746 12502 raft_consensus.cc:348] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:31.717619 12502 raft_consensus.cc:729] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Initialized, Role: FOLLOWER
I20260806 01:10:31.718287 12502 consensus_queue.cc:260] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:31.718547 12502 raft_consensus.cc:388] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:31.718698 12502 raft_consensus.cc:482] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:31.718889 12502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:10:31.722863 12502 raft_consensus.cc:504] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:31.723419 12502 leader_election.cc:302] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d190f8b3787f411291d7c95f910cb110; no voters:
I20260806 01:10:31.724146 12502 leader_election.cc:288] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:10:31.724299 12507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:10:31.725634 12507 raft_consensus.cc:686] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 LEADER]: Becoming Leader. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Running, Role: LEADER
I20260806 01:10:31.726308 12507 consensus_queue.cc:237] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:31.726634 12502 sys_catalog.cc:564] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:31.728991 12509 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d190f8b3787f411291d7c95f910cb110. Latest consensus state: current_term: 2 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:31.729439 12509 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:31.730068 12508 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:31.730410 12508 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:31.737038 12513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:31.745369 12513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:31.747098 12513 catalog_manager.cc:1259] Loaded cluster ID: f0b1a5af6a844bdb88477a23bbff2b88
I20260806 01:10:31.747263 12513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:31.750589 12513 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 01:10:31.768877 12513 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 01:10:31.794015 12513 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:31.796227 12513 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Loaded TSK: 0
I20260806 01:10:31.797823 12513 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:10:32.045439 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:32.045761 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:32.045940 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:32.053442 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:32.053599 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:32.053771 12505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:32.064467 12505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:44967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=41331
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:32.065884 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:32.067282 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:32.083039 12540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:32.083272 12538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:32.083676 12544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:32.089454 12505 server_base.cc:1034] running on GCE node
I20260806 01:10:32.090155 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:32.091216 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:32.092460 12505 hybrid_clock.cc:648] HybridClock initialized: now 1785978632092397 us; error 63 us; skew 500 ppm
I20260806 01:10:32.092803 12505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:32 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:32.096474 12505 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:32.098172 12505 webserver.cc:460] Webserver started at http://127.22.63.65:41331/ using document root <none> and password file <none>
I20260806 01:10:32.098901 12505 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:32.099063 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:32.104734 12505 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:10:32.107834 12553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.109316 12505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:10:32.109504 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "d167b7a4e4294d679b5e0d970d75ce39"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "cc63e003c9836dcab39dc8b5be25985b"
server_key_iv: "77fc0376f0be552b8c157845128e0296"
server_key_version: "encryptionkey@0"
I20260806 01:10:32.110509 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:32.132182 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:32.136888 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:32.137203 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:32.138384 12505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:32.139925 12505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:32.140049 12505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.140198 12505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:32.140290 12505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.175912 12505 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44967
I20260806 01:10:32.175949 12730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44967 every 8 connection(s)
I20260806 01:10:32.177486 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:32.185429 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 12505
I20260806 01:10:32.186141 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:33541
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=36771
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:32 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:32.202378 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:32.180840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:35495)
0806 01:10:32.181650 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:32.181686 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:32.183391 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:32.184757 (+ 1366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:32.184771 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:32.185419 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:32.185983 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:32.186000 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.187772 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.187779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:32.189027 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:32.189039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.190074 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.190082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:32.190354 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:32.191078 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:32.191106 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:32.193280 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:32.197918 (+ 4638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:32.197931 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:32.197951 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:32.198664 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:32.199232 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:32.199235 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:32.199238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:32.199433 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:32.200437 (+ 1004us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:32.200442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:32.200727 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 01:10:32.201005 (+ 278us) client_negotiation.cc:240] Negotiation successful
0806 01:10:32.201387 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":356,"threads_started":1}
I20260806 01:10:32.202901 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:32.180522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:35495 (local address 127.22.63.126:43753)
0806 01:10:32.181217 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:32.181234 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:32.181824 (+ 590us) server_negotiation.cc:407] Connection header received
0806 01:10:32.183966 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:32.183977 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:32.184269 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:32.184623 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:32.186213 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.187564 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.189173 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.189820 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.190026 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:32.193562 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:32.193570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:32.193583 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:32.193626 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:32.197755 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:32.198826 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:32.198837 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:32.198846 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:32.199102 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:32.199557 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:32.199563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:32.199568 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:32.200278 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:32.200506 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:32.201255 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 01:10:32.201698 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":328,"threads_started":1}
I20260806 01:10:32.205600 12731 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:32.205914 12731 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:32.206898 12731 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:32.209362 12445 ts_manager.cc:194] Registered new tserver with Master: d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65:44967)
I20260806 01:10:32.210884 12445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:35495
W20260806 01:10:32.543514 12741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:32.543922 12741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:32.544086 12741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:32.551694 12741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:32.551872 12741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:32.552078 12741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:32.562065 12741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:33541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=36771
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:32.563489 12741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:32.564908 12741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:32.579164 12746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:32.579192 12747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:32.579244 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:32.580356 12741 server_base.cc:1034] running on GCE node
I20260806 01:10:32.581039 12741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:32.582104 12741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:32.583335 12741 hybrid_clock.cc:648] HybridClock initialized: now 1785978632583278 us; error 47 us; skew 500 ppm
I20260806 01:10:32.583636 12741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:32 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:32.587000 12741 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:32.588601 12741 webserver.cc:460] Webserver started at http://127.22.63.66:36771/ using document root <none> and password file <none>
I20260806 01:10:32.589358 12741 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:32.589535 12741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:32.595104 12741 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 01:10:32.598078 12757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.599480 12741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:10:32.599679 12741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "4256d56e9bb2407b9c41380efe3c3a44"
format_stamp: "Formatted at 2026-08-06 01:10:29 on dist-test-slave-7f01"
server_key: "db43df2679343b0a7c5ed065d0178e15"
server_key_iv: "a8f615c5a3cd1d3ba26188ad586d2bac"
server_key_version: "encryptionkey@0"
I20260806 01:10:32.600955 12741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:32.621774 12741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:32.626233 12741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:32.626541 12741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:32.627964 12741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:32.629699 12741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:32.629802 12741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.629953 12741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:32.630069 12741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:32.676007 12741 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:33541
I20260806 01:10:32.676028 12931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:33541 every 8 connection(s)
I20260806 01:10:32.677556 12741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:32.682765 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 12741
I20260806 01:10:32.683386 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:33739
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=37053
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:10:32 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:32.699055 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:32.680332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:43513 (local address 127.22.63.126:43753)
0806 01:10:32.680534 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:32.680538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:32.682405 (+ 1867us) server_negotiation.cc:407] Connection header received
0806 01:10:32.684485 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:32.684491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:32.684605 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:32.684721 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:32.686256 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.686977 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.688614 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.689055 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.689190 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:32.692735 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:32.692740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:32.692743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:32.692777 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:32.695638 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:32.696573 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:32.696583 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:32.696585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:32.696741 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:32.697210 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:32.697215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:32.697217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:32.697506 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:32.697680 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:32.698427 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 01:10:32.698669 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:10:32.700222 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:32.680924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:43513)
0806 01:10:32.682250 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:32.682287 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:32.684209 (+ 1922us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:32.684977 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:32.684986 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:32.685509 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:32.686074 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:32.686091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.687151 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.687161 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:32.688447 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:32.688459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:32.689311 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:32.689318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:32.689557 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:32.690256 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:32.690284 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:32.692509 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:32.695815 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:32.695825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:32.695843 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:32.696428 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:32.696871 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:32.696876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:32.696880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:32.697076 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:32.697645 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:32.697654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:32.698033 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 01:10:32.698312 (+ 279us) client_negotiation.cc:240] Negotiation successful
0806 01:10:32.698899 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":816,"threads_started":1}
I20260806 01:10:32.701879 12933 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:32.702275 12933 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:32.703303 12933 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:32.705204 12445 ts_manager.cc:194] Registered new tserver with Master: 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541)
I20260806 01:10:32.706525 12445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:43513
W20260806 01:10:33.049665 12939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:33.049952 12939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:33.050046 12939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:33.057796 12939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:33.057919 12939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:33.058034 12939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:33.067958 12939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:33739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=37053
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:33.069244 12939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:33.070562 12939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:33.084002 12955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:33.084086 12950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:33.084167 12949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:33.085189 12939 server_base.cc:1034] running on GCE node
I20260806 01:10:33.085867 12939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:33.086882 12939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:33.088120 12939 hybrid_clock.cc:648] HybridClock initialized: now 1785978633088083 us; error 38 us; skew 500 ppm
I20260806 01:10:33.088455 12939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:33 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:33.091689 12939 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:33.093274 12939 webserver.cc:460] Webserver started at http://127.22.63.67:37053/ using document root <none> and password file <none>
I20260806 01:10:33.094004 12939 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:33.094172 12939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:33.100039 12939 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:10:33.102859 12966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:33.104557 12939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:10:33.104758 12939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
format_stamp: "Formatted at 2026-08-06 01:10:30 on dist-test-slave-7f01"
server_key: "9a5d931d0fe449276e6609b3ee465b87"
server_key_iv: "781fe1d371efa062e86a7401f14bcb44"
server_key_version: "encryptionkey@0"
I20260806 01:10:33.105931 12939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:33.132185 12939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:33.136677 12939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:33.137044 12939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:33.138212 12939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:33.139685 12939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:33.139848 12939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 01:10:33.139982 12939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:33.140077 12939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:33.176815 12939 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:33739
I20260806 01:10:33.176877 13149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:33739 every 8 connection(s)
I20260806 01:10:33.178304 12939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:33.188117 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 12939
Aug 06 01:10:33 dist-test-slave-7f01 krb5kdc[11463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:33.199970 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.181259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:51549 (local address 127.22.63.126:43753)
0806 01:10:33.181496 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.181503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.182758 (+ 1255us) server_negotiation.cc:407] Connection header received
0806 01:10:33.184790 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.184795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.184863 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.184951 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:33.186454 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.187194 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.188977 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.189640 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.189800 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.193752 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:33.193757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:33.193760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:33.193789 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:33.196790 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.197544 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.197549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.197551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.197666 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.198106 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.198113 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.198120 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.198407 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:33.198586 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.199339 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.199621 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:10:33.200953 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.181713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:51549)
0806 01:10:33.182583 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.182634 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.184550 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.185154 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.185162 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.185697 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:33.186257 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.186285 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.187336 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.187340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.188806 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.188825 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.189928 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.189936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.190209 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.191055 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:33.191078 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:33.193544 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:33.196948 (+ 3404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:33.196957 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:33.196973 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:33.197406 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:33.197781 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:33.197786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:33.197790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:33.197978 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:33.198557 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:33.198564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:33.198942 (+ 378us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.199207 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.199780 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":320,"threads_started":1}
I20260806 01:10:33.202409 13150 heartbeater.cc:347] Connected to a master server at 127.22.63.126:43753
I20260806 01:10:33.202732 13150 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:33.203513 13150 heartbeater.cc:511] Master 127.22.63.126:43753 requested a full tablet report, sending...
I20260806 01:10:33.204207 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.189131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.22.63.126:43753)
0806 01:10:33.189679 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.189686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.190214 (+ 528us) server_negotiation.cc:407] Connection header received
0806 01:10:33.190358 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.190368 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.190463 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.190590 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:33.192284 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.193244 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.194803 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.195287 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.195432 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.198145 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:33.198150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:33.198154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:33.198189 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:33.201253 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.202086 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.202093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.202098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.202227 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.202645 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.202651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.202654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.202988 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:33.203257 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.203561 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.203829 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":325,"threads_started":1}
I20260806 01:10:33.205256 12445 ts_manager.cc:194] Registered new tserver with Master: 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739)
I20260806 01:10:33.206503 12445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:51549
I20260806 01:10:33.207065 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:10:33.214041 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.208118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33484 (local address 127.22.63.126:43753)
0806 01:10:33.208282 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.208288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.208674 (+ 386us) server_negotiation.cc:407] Connection header received
0806 01:10:33.208861 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.208866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.209029 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.209223 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:10:33.209796 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.210493 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.212053 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.212467 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.212562 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.213000 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:10:33.213469 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:10:33.213569 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.213763 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.213850 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:10:33.214936 12731 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:33.215075 12445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:10:33.222772 12445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:10:33.245538 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.234873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.22.63.65:44967)
0806 01:10:33.235490 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.235500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.235855 (+ 355us) server_negotiation.cc:407] Connection header received
0806 01:10:33.235995 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.236002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.236289 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.236513 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:33.237327 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.239449 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.242471 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.244232 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.244367 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.244634 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.244962 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.245191 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":730,"threads_started":1}
I20260806 01:10:33.245826 13168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.234673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44967 (local address 127.0.0.1:48074)
0806 01:10:33.235550 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.235587 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.235789 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.236779 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.236787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.236826 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:33.237129 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.237138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.239609 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.239621 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.242299 (+ 2678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.242317 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.244480 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.244487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.244690 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.244744 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.244909 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.245122 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":53,"thread_start_us":488,"threads_started":1}
I20260806 01:10:33.246356 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.236334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.22.63.66:33541)
0806 01:10:33.238589 (+ 2255us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.238597 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.238628 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:10:33.238729 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.238737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.239036 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.239242 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:33.240004 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.241210 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.243549 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.244909 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.245036 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.245211 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.245838 (+ 627us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.246040 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2064,"thread_start_us":1946,"threads_started":1}
I20260806 01:10:33.248122 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.237092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.22.63.67:33739)
0806 01:10:33.237771 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.237779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.238805 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 01:10:33.239043 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.239050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.239267 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.239464 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:33.240451 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.242035 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.244780 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.246211 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.246366 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.246592 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.247602 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.247806 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":488,"threads_started":1}
I20260806 01:10:33.248200 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.237448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33739 (local address 127.0.0.1:54596)
0806 01:10:33.238699 (+ 1251us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.238717 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.238843 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.239523 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.239529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.239559 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:33.240196 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.240210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.242254 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.242263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.244588 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.244607 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.246971 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.246978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.247149 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.247181 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.247303 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.247449 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":22,"thread_start_us":479,"threads_started":1}
I20260806 01:10:33.248224 13171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.236013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:33541 (local address 127.0.0.1:46614)
0806 01:10:33.236632 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.236653 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.236765 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.239244 (+ 2479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.239249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.239275 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:33.239800 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.239815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.241366 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.241373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.243397 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.243409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.245390 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.245396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.245602 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.245650 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.246498 (+ 848us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.246647 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":1,"thread_start_us":331,"threads_started":1}
I20260806 01:10:33.250447 12631 tablet_service.cc:1467] Processing CreateTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 (DEFAULT_TABLE table=test-table [id=386e9630c14b484ca9735951f896b4b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:33.251291 13044 tablet_service.cc:1467] Processing CreateTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 (DEFAULT_TABLE table=test-table [id=386e9630c14b484ca9735951f896b4b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:33.252045 12631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4744460c5ab7489a802325a0cbd5edc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:33.251780 12831 tablet_service.cc:1467] Processing CreateTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 (DEFAULT_TABLE table=test-table [id=386e9630c14b484ca9735951f896b4b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:33.252678 13044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4744460c5ab7489a802325a0cbd5edc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:33.253049 12831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4744460c5ab7489a802325a0cbd5edc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:33.264997 13178 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Bootstrap starting.
I20260806 01:10:33.267660 13179 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Bootstrap starting.
I20260806 01:10:33.268702 13178 tablet_bootstrap.cc:654] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:33.269248 13180 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Bootstrap starting.
I20260806 01:10:33.271368 13179 tablet_bootstrap.cc:654] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:33.271509 13178 log.cc:824] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:33.272312 13180 tablet_bootstrap.cc:654] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:33.272665 13179 log.cc:824] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:33.273617 13180 log.cc:824] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:33.274974 13178 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: No bootstrap required, opened a new log
I20260806 01:10:33.275282 13178 ts_tablet_manager.cc:1397] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 01:10:33.275727 13179 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: No bootstrap required, opened a new log
I20260806 01:10:33.276053 13179 ts_tablet_manager.cc:1397] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 01:10:33.276357 13180 tablet_bootstrap.cc:492] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: No bootstrap required, opened a new log
I20260806 01:10:33.276607 13180 ts_tablet_manager.cc:1397] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 01:10:33.284498 13180 raft_consensus.cc:348] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.284910 13180 raft_consensus.cc:374] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:33.285040 13180 raft_consensus.cc:729] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4256d56e9bb2407b9c41380efe3c3a44, State: Initialized, Role: FOLLOWER
I20260806 01:10:33.285691 13180 consensus_queue.cc:260] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.286875 12933 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:33.286974 13178 raft_consensus.cc:348] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.286973 13179 raft_consensus.cc:348] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.287436 13179 raft_consensus.cc:374] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:33.287442 13178 raft_consensus.cc:374] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:33.287547 13179 raft_consensus.cc:729] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d167b7a4e4294d679b5e0d970d75ce39, State: Initialized, Role: FOLLOWER
I20260806 01:10:33.287547 13178 raft_consensus.cc:729] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca9c8dc55c4cbbb0ee4193917e823b, State: Initialized, Role: FOLLOWER
I20260806 01:10:33.288304 13179 consensus_queue.cc:260] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.288300 13178 consensus_queue.cc:260] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.289448 13180 ts_tablet_manager.cc:1428] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:10:33.295698 13150 heartbeater.cc:503] Master 127.22.63.126:43753 was elected leader, sending a full tablet report...
I20260806 01:10:33.296878 13179 ts_tablet_manager.cc:1428] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
I20260806 01:10:33.296882 13178 ts_tablet_manager.cc:1428] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Time spent starting tablet: real 0.021s user 0.013s sys 0.008s
I20260806 01:10:33.318722 13190 raft_consensus.cc:482] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:10:33.319099 13190 raft_consensus.cc:504] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.320851 13190 leader_election.cc:288] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541), 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739)
I20260806 01:10:33.329313 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.321485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:35463 (local address 127.22.63.66:33541)
0806 01:10:33.321706 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.321711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.322222 (+ 511us) server_negotiation.cc:407] Connection header received
0806 01:10:33.322525 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.322532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.322618 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.322691 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:33.323337 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.324405 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.327015 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.328313 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.328473 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.328553 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.328936 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.329031 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:10:33.329471 13191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.321311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:33541 (local address 127.22.63.65:35463)
0806 01:10:33.322096 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.322115 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.322260 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.322868 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.322871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.322892 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:33.323194 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.323201 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.324559 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.324563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.326863 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.326878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.328526 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.328530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.328741 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.328797 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.328898 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.329032 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":4,"thread_start_us":403,"threads_started":1}
I20260806 01:10:33.329623 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.322179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:33739 (local address 127.22.63.65:42837)
0806 01:10:33.322777 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:33.322795 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:33.322999 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:33.323380 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:33.323383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:33.323404 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:33.323682 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.323689 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.324947 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.324952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.327075 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:33.327085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.328740 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.328745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:33.328990 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.329020 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:10:33.329120 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:10:33.329246 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":30,"thread_start_us":292,"threads_started":1}
I20260806 01:10:33.329782 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.322534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:42837 (local address 127.22.63.67:33739)
0806 01:10:33.322744 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.322749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.322888 (+ 139us) server_negotiation.cc:407] Connection header received
0806 01:10:33.323165 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.323169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.323229 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.323338 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:33.323890 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.324808 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.327211 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.328527 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.328672 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.328721 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.329153 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.329276 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:10:33.330693 13072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4744460c5ab7489a802325a0cbd5edc2" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" is_pre_election: true
I20260806 01:10:33.330701 12866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4744460c5ab7489a802325a0cbd5edc2" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4256d56e9bb2407b9c41380efe3c3a44" is_pre_election: true
I20260806 01:10:33.331213 12866 raft_consensus.cc:2454] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 0.
I20260806 01:10:33.331211 13072 raft_consensus.cc:2454] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 0.
I20260806 01:10:33.331936 12554 leader_election.cc:302] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4256d56e9bb2407b9c41380efe3c3a44, d167b7a4e4294d679b5e0d970d75ce39; no voters:
I20260806 01:10:33.332335 13190 raft_consensus.cc:2781] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:10:33.332496 13190 raft_consensus.cc:482] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:10:33.332615 13190 raft_consensus.cc:3037] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:33.334867 13190 raft_consensus.cc:504] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.335548 13190 leader_election.cc:288] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 election: Requested vote from peers 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541), 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739)
I20260806 01:10:33.336033 12866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4744460c5ab7489a802325a0cbd5edc2" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4256d56e9bb2407b9c41380efe3c3a44"
I20260806 01:10:33.336113 13072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4744460c5ab7489a802325a0cbd5edc2" candidate_uuid: "d167b7a4e4294d679b5e0d970d75ce39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
I20260806 01:10:33.336228 12866 raft_consensus.cc:3037] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:33.336351 13072 raft_consensus.cc:3037] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:33.339090 12866 raft_consensus.cc:2454] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 1.
I20260806 01:10:33.339224 13072 raft_consensus.cc:2454] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d167b7a4e4294d679b5e0d970d75ce39 in term 1.
I20260806 01:10:33.339588 12554 leader_election.cc:302] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4256d56e9bb2407b9c41380efe3c3a44, d167b7a4e4294d679b5e0d970d75ce39; no voters:
I20260806 01:10:33.339901 13190 raft_consensus.cc:2781] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:33.340365 13190 raft_consensus.cc:686] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 1 LEADER]: Becoming Leader. State: Replica: d167b7a4e4294d679b5e0d970d75ce39, State: Running, Role: LEADER
I20260806 01:10:33.340839 13190 consensus_queue.cc:237] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } }
I20260806 01:10:33.345885 12444 catalog_manager.cc:5495] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 reported cstate change: term changed from 0 to 1, leader changed from <none> to d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "d167b7a4e4294d679b5e0d970d75ce39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d167b7a4e4294d679b5e0d970d75ce39" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 } health_report { overall_health: UNKNOWN } } }
I20260806 01:10:33.375488 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.370002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.22.63.65:44967)
0806 01:10:33.370220 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.370224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.370242 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:10:33.370343 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.370347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.370439 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.370570 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:10:33.370936 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.371666 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.373315 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.373732 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.373846 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.374425 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:10:33.374942 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:10:33.375064 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.375186 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.375268 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 01:10:33.384202 13072 raft_consensus.cc:1261] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Refusing update from remote peer d167b7a4e4294d679b5e0d970d75ce39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:33.384316 12866 raft_consensus.cc:1261] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Refusing update from remote peer d167b7a4e4294d679b5e0d970d75ce39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:33.385250 13194 consensus_queue.cc:1035] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53ca9c8dc55c4cbbb0ee4193917e823b" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 33739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:33.385649 13190 consensus_queue.cc:1035] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4256d56e9bb2407b9c41380efe3c3a44" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:33.416219 12445 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:33.416551 12445 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:33.420423 12445 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Sending DeleteTablet for 3 replicas of tablet 4744460c5ab7489a802325a0cbd5edc2
I20260806 01:10:33.421692 12631 tablet_service.cc:1514] Processing DeleteTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:33 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:48074
I20260806 01:10:33.421895 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 12505
I20260806 01:10:33.421816 13044 tablet_service.cc:1514] Processing DeleteTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:33 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:54596
I20260806 01:10:33.422304 12831 tablet_service.cc:1514] Processing DeleteTablet for tablet 4744460c5ab7489a802325a0cbd5edc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:33 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46614
I20260806 01:10:33.435999 13209 tablet_replica.cc:330] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: stopping tablet replica
I20260806 01:10:33.436694 13213 tablet_replica.cc:330] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: stopping tablet replica
I20260806 01:10:33.437242 13213 raft_consensus.cc:2229] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:33.437805 13209 raft_consensus.cc:2229] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:10:33.438488 13213 raft_consensus.cc:2258] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:33.439291 13209 raft_consensus.cc:2258] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:33.441803 13209 ts_tablet_manager.cc:1905] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:33.441792 13213 ts_tablet_manager.cc:1905] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:33.448945 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.422774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.22.63.65:44967)
0806 01:10:33.423013 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.423021 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.423336 (+ 315us) server_negotiation.cc:407] Connection header received
0806 01:10:33.423417 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.423425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.423542 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.423650 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:33.426273 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.428024 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.431620 (+ 3596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.432583 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.432738 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.437761 (+ 5023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:33.437785 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:33.437811 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:33.437879 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:33.443193 (+ 5314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.444675 (+ 1482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.444688 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.444694 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.444940 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.445582 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.445590 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.445596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.446236 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:33.446659 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.448054 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.448399 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:10:33.449188 13213 ts_tablet_manager.cc:1918] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:33.449404 13213 log.cc:1196] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/4744460c5ab7489a802325a0cbd5edc2
I20260806 01:10:33.449188 13209 ts_tablet_manager.cc:1918] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:33.449744 13209 log.cc:1196] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/4744460c5ab7489a802325a0cbd5edc2
I20260806 01:10:33.449867 13211 tablet_replica.cc:330] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: stopping tablet replica
I20260806 01:10:33.450444 13209 ts_tablet_manager.cc:1939] T 4744460c5ab7489a802325a0cbd5edc2 P d167b7a4e4294d679b5e0d970d75ce39: Deleting consensus metadata
I20260806 01:10:33.450493 12721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:33.450444 13211 raft_consensus.cc:2229] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:33.451079 13211 raft_consensus.cc:2258] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:33.451496 13213 ts_tablet_manager.cc:1939] T 4744460c5ab7489a802325a0cbd5edc2 P 4256d56e9bb2407b9c41380efe3c3a44: Deleting consensus metadata
I20260806 01:10:33.453501 13211 ts_tablet_manager.cc:1905] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:33.453457 12412 catalog_manager.cc:4841] TS 4256d56e9bb2407b9c41380efe3c3a44 (127.22.63.66:33541): tablet 4744460c5ab7489a802325a0cbd5edc2 (table test-table [id=386e9630c14b484ca9735951f896b4b2]) successfully deleted
I20260806 01:10:33.459414 13211 ts_tablet_manager.cc:1918] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:33.459607 13211 log.cc:1196] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/4744460c5ab7489a802325a0cbd5edc2
I20260806 01:10:33.460114 13211 ts_tablet_manager.cc:1939] T 4744460c5ab7489a802325a0cbd5edc2 P 53ca9c8dc55c4cbbb0ee4193917e823b: Deleting consensus metadata
I20260806 01:10:33.461652 12412 catalog_manager.cc:4841] TS 53ca9c8dc55c4cbbb0ee4193917e823b (127.22.63.67:33739): tablet 4744460c5ab7489a802325a0cbd5edc2 (table test-table [id=386e9630c14b484ca9735951f896b4b2]) successfully deleted
I20260806 01:10:33.549973 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 12505
I20260806 01:10:33.551460 12412 catalog_manager.cc:4841] TS d167b7a4e4294d679b5e0d970d75ce39 (127.22.63.65:44967): tablet 4744460c5ab7489a802325a0cbd5edc2 (table test-table [id=386e9630c14b484ca9735951f896b4b2]) successfully deleted
I20260806 01:10:33.574870 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 12741
I20260806 01:10:33.590008 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.575683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.22.63.66:33541)
0806 01:10:33.575893 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.575900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.575990 (+ 90us) server_negotiation.cc:407] Connection header received
0806 01:10:33.576211 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.576216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.576304 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.576423 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:33.577829 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.578631 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.580153 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.580578 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.580724 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.583684 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:33.583701 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:33.583719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:33.583901 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:33.586923 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.587726 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.587763 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.587769 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.587919 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.588336 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.588341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.588345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.588753 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:33.589072 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.589397 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.589641 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 01:10:33.590705 12926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:33.682190 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 12741
I20260806 01:10:33.707087 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 12939
I20260806 01:10:33.721609 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:33.708006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54606 (local address 127.22.63.67:33739)
0806 01:10:33.708210 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:33.708216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:33.708285 (+ 69us) server_negotiation.cc:407] Connection header received
0806 01:10:33.708503 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:33.708509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:33.708591 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:33.708704 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:33.710230 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.711079 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.712610 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:33.713064 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:33.713209 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:33.715958 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:33.715976 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:33.715993 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:33.716032 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:33.718714 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.719505 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.719512 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.719515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.719662 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:33.720115 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:33.720118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:33.720121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:33.720525 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:33.720755 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:33.721033 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 01:10:33.721223 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:10:33.722262 13144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:33.812880 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 12939
I20260806 01:10:33.837893 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 12384
I20260806 01:10:33.839267 12494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:33.959551 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 12384
I20260806 01:10:33.991163 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--webserver_interface=127.22.63.126
--webserver_port=44921
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:43753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:34.350903 13224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:34.351248 13224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:34.351424 13224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:34.358620 13224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:34.358776 13224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:34.358891 13224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:34.358971 13224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:34.359040 13224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:34.368991 13224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:43753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:43753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=44921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:34.370477 13224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:34.371855 13224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:34.384091 13232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:34.384136 13235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:34.384114 13238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:34.385473 13224 server_base.cc:1034] running on GCE node
I20260806 01:10:34.386332 13224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:34.387874 13224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:34.389120 13224 hybrid_clock.cc:648] HybridClock initialized: now 1785978634389051 us; error 60 us; skew 500 ppm
I20260806 01:10:34.389510 13224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:34 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978634, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:34.392666 13224 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:34.394222 13224 webserver.cc:460] Webserver started at http://127.22.63.126:44921/ using document root <none> and password file <none>
I20260806 01:10:34.394949 13224 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:34.395107 13224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:34.400931 13224 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:10:34.403820 13246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:34.405090 13224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:10:34.405282 13224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "d190f8b3787f411291d7c95f910cb110"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "8204dac3e31657c0a776b5d850238d25"
server_key_iv: "aa7c11a1ec259173b344d2019fb470a8"
server_key_version: "encryptionkey@0"
I20260806 01:10:34.406293 13224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:34.427099 13224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:34.431249 13224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:34.431557 13224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:34.450973 13224 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:43753
I20260806 01:10:34.450994 13318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:43753 every 8 connection(s)
I20260806 01:10:34.452669 13224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:34.459724 13319 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:10:34.461374 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 13224
I20260806 01:10:34.462442 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:44967
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=41331
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:34.464459 13319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap starting.
I20260806 01:10:34.488797 13319 log.cc:824] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:34.505259 13319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:10:34.505849 13319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110: Bootstrap complete.
I20260806 01:10:34.515805 13319 raft_consensus.cc:348] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:34.516728 13319 raft_consensus.cc:729] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Initialized, Role: FOLLOWER
I20260806 01:10:34.517379 13319 consensus_queue.cc:260] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:34.517640 13319 raft_consensus.cc:388] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:34.517763 13319 raft_consensus.cc:482] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:34.517962 13319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:10:34.521816 13319 raft_consensus.cc:504] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:34.522364 13319 leader_election.cc:302] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d190f8b3787f411291d7c95f910cb110; no voters:
I20260806 01:10:34.523047 13319 leader_election.cc:288] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:10:34.523195 13331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:10:34.524636 13331 raft_consensus.cc:686] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [term 3 LEADER]: Becoming Leader. State: Replica: d190f8b3787f411291d7c95f910cb110, State: Running, Role: LEADER
I20260806 01:10:34.525447 13319 sys_catalog.cc:564] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:34.525287 13331 consensus_queue.cc:237] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } }
I20260806 01:10:34.528121 13332 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:34.528554 13332 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:34.530867 13333 sys_catalog.cc:455] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d190f8b3787f411291d7c95f910cb110. Latest consensus state: current_term: 3 leader_uuid: "d190f8b3787f411291d7c95f910cb110" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d190f8b3787f411291d7c95f910cb110" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 43753 } } }
I20260806 01:10:34.531273 13333 sys_catalog.cc:458] T 00000000000000000000000000000000 P d190f8b3787f411291d7c95f910cb110 [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:34.533100 13337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:34.543291 13337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:34.544868 13337 catalog_manager.cc:1259] Loaded cluster ID: f0b1a5af6a844bdb88477a23bbff2b88
I20260806 01:10:34.545051 13337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 01:10:34.547330 13337 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f70c645e9ad google::LogMessage::Fail() at ??:0
@ 0x7f70c646188a google::LogMessage::SendToLog() at ??:0
@ 0x7f70c645e4de google::LogMessage::Flush() at ??:0
@ 0x7f70c645fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f70dbe91512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f70dbe8e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f70dbf16610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f70dbf16439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f70c90aa345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f70c90ad91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f70c90ad799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f70c9088eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f70cc3626db start_thread at ??:0
@ 0x7f70c52f861f clone at ??:0
W20260806 01:10:34.899977 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:34.900277 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:34.900378 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:34.907858 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:34.907985 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:34.908102 13325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:34.917972 13325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:44967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=41331
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:34.919281 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:34.920624 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:34.934442 13389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:34.934422 13394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:34.934370 13391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:34.935178 13325 server_base.cc:1034] running on GCE node
I20260806 01:10:34.935904 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:34.936888 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:34.938119 13325 hybrid_clock.cc:648] HybridClock initialized: now 1785978634938072 us; error 39 us; skew 500 ppm
I20260806 01:10:34.938493 13325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:34 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978634, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:34.941905 13325 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:34.943565 13325 webserver.cc:460] Webserver started at http://127.22.63.65:41331/ using document root <none> and password file <none>
I20260806 01:10:34.944357 13325 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:34.944514 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:34.950215 13325 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 01:10:34.953316 13405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:34.954802 13325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:10:34.955024 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "d167b7a4e4294d679b5e0d970d75ce39"
format_stamp: "Formatted at 2026-08-06 01:10:28 on dist-test-slave-7f01"
server_key: "cc63e003c9836dcab39dc8b5be25985b"
server_key_iv: "77fc0376f0be552b8c157845128e0296"
server_key_version: "encryptionkey@0"
I20260806 01:10:34.956166 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:34.987521 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:34.992375 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:34.992700 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:34.993805 13325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:34.995326 13325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:34.995462 13325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:34.995605 13325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:34.995702 13325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.032295 13325 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44967
I20260806 01:10:35.032397 13576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44967 every 8 connection(s)
I20260806 01:10:35.033836 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:35.038882 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.036996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:60211)
0806 01:10:35.037746 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.037991 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":305,"threads_started":1}
W20260806 01:10:35.039635 13577 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:43753 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:43753: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
I20260806 01:10:35.040512 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.040194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:42755)
0806 01:10:35.040337 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.040406 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:10:35.041600 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.041060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:34537)
0806 01:10:35.041324 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.041444 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260806 01:10:35.041886 13577 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:10:35.041898 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 13325
I20260806 01:10:35.042543 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:33541
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=36771
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:35.392628 13580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:35.392926 13580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:35.393132 13580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:35.400683 13580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:35.400804 13580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:35.401028 13580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:35.411010 13580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:33541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=36771
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:35.412531 13580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:35.413928 13580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:35.427896 13589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:35.427862 13591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:35.427961 13588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:35.433683 13580 server_base.cc:1034] running on GCE node
I20260806 01:10:35.434427 13580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:35.435487 13580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:35.436717 13580 hybrid_clock.cc:648] HybridClock initialized: now 1785978635436663 us; error 41 us; skew 500 ppm
I20260806 01:10:35.437076 13580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:35 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978635, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:35.440431 13580 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:35.442024 13580 webserver.cc:460] Webserver started at http://127.22.63.66:36771/ using document root <none> and password file <none>
I20260806 01:10:35.442803 13580 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:35.442914 13580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:35.448570 13580 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:10:35.451247 13605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.452781 13580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:10:35.452965 13580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "4256d56e9bb2407b9c41380efe3c3a44"
format_stamp: "Formatted at 2026-08-06 01:10:29 on dist-test-slave-7f01"
server_key: "db43df2679343b0a7c5ed065d0178e15"
server_key_iv: "a8f615c5a3cd1d3ba26188ad586d2bac"
server_key_version: "encryptionkey@0"
I20260806 01:10:35.453979 13580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:35.468238 13580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:35.472441 13580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:35.472741 13580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:35.473799 13580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:35.475116 13580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:35.475214 13580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.475294 13580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:35.475358 13580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.511205 13580 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:33541
I20260806 01:10:35.511303 13780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:33541 every 8 connection(s)
I20260806 01:10:35.512818 13580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:35.514124 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 13580
I20260806 01:10:35.514927 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:33739
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:43753
--webserver_port=37053
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:46425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:10:35.518671 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.516305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:35229)
0806 01:10:35.517225 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.517513 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":318,"threads_started":1}
W20260806 01:10:35.519405 13782 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:43753 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:43753: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
I20260806 01:10:35.520416 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.520058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:57257)
0806 01:10:35.520217 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.520296 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:10:35.521370 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:35.521040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:47067)
0806 01:10:35.521174 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:35.521250 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:10:35.521669 13782 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:10:35.866748 13788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:35.867072 13788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:35.867170 13788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:35.874398 13788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:35.874521 13788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:35.874634 13788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:35.889099 13788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:46425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:33739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=37053
--tserver_master_addrs=127.22.63.126:43753
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:35.890437 13788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:35.891868 13788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:35.906419 13794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:35.906306 13796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:35.906315 13798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:35.907514 13788 server_base.cc:1034] running on GCE node
I20260806 01:10:35.908216 13788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:35.909190 13788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:35.910424 13788 hybrid_clock.cc:648] HybridClock initialized: now 1785978635910381 us; error 39 us; skew 500 ppm
I20260806 01:10:35.910768 13788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:35 dist-test-slave-7f01 krb5kdc[11463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978635, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:35.914058 13788 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:35.915627 13788 webserver.cc:460] Webserver started at http://127.22.63.67:37053/ using document root <none> and password file <none>
I20260806 01:10:35.916399 13788 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:35.916565 13788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:35.922009 13788 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 01:10:35.924891 13811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.926189 13788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:10:35.926369 13788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "53ca9c8dc55c4cbbb0ee4193917e823b"
format_stamp: "Formatted at 2026-08-06 01:10:30 on dist-test-slave-7f01"
server_key: "9a5d931d0fe449276e6609b3ee465b87"
server_key_iv: "781fe1d371efa062e86a7401f14bcb44"
server_key_version: "encryptionkey@0"
I20260806 01:10:35.927616 13788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:35.966369 13788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:35.970737 13788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:35.971062 13788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:35.972141 13788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:35.973580 13788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:35.973717 13788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:35.973877 13788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:35.973973 13788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:36.009843 13788 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:33739
I20260806 01:10:36.009941 13975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:33739 every 8 connection(s)
I20260806 01:10:36.011420 13788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:36.016458 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:36.014628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:39317)
0806 01:10:36.015361 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:36.015591 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":284,"threads_started":1}
W20260806 01:10:36.017210 13976 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:43753 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:43753: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
I20260806 01:10:36.017975 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:36.017672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:38331)
0806 01:10:36.017804 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:36.017873 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 01:10:36.018811 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:36.018516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:40179)
0806 01:10:36.018637 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:36.018709 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 01:10:36.019136 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 13788
W20260806 01:10:36.019318 13976 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:10:36.043442 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:36.042476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:47037)
0806 01:10:36.043061 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:36.043178 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":280,"threads_started":1}
I20260806 01:10:36.523272 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:36.522380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:35103)
0806 01:10:36.522908 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:36.523031 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":260,"threads_started":1}
I20260806 01:10:37.020710 13982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:37.019917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:36365)
0806 01:10:37.020378 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:37.020473 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":222,"threads_started":1}
I20260806 01:10:37.045202 13983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:37.044414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:37193)
0806 01:10:37.044935 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:37.045021 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":250,"threads_started":1}
I20260806 01:10:37.525463 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:37.524395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:34593)
0806 01:10:37.525047 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:37.525178 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":270,"threads_started":1}
I20260806 01:10:38.022524 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:38.021728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:51451)
0806 01:10:38.022239 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:38.022329 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":255,"threads_started":1}
I20260806 01:10:38.046938 13993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:38.046204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:60227)
0806 01:10:38.046659 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:38.046739 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":255,"threads_started":1}
I20260806 01:10:38.527165 13995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:38.526385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:40815)
0806 01:10:38.526911 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:38.526988 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":307,"threads_started":1}
I20260806 01:10:39.024219 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:39.023458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:34845)
0806 01:10:39.023959 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:39.024039 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":240,"threads_started":1}
I20260806 01:10:39.048589 13997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:39.047870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:58053)
0806 01:10:39.048320 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:39.048403 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260806 01:10:39.528970 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:39.528159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:57349)
0806 01:10:39.528667 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:39.528765 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260806 01:10:40.025866 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:40.025137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:36711)
0806 01:10:40.025615 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:40.025698 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":261,"threads_started":1}
I20260806 01:10:40.050314 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:40.049541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:47379)
0806 01:10:40.050047 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:40.050131 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":268,"threads_started":1}
I20260806 01:10:40.530678 14003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:40.529936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:60399)
0806 01:10:40.530400 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:40.530486 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":240,"threads_started":1}
I20260806 01:10:41.027642 14007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:41.026893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:53385)
0806 01:10:41.027390 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:41.027474 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":257,"threads_started":1}
I20260806 01:10:41.052088 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:41.051248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:43919)
0806 01:10:41.051836 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:41.051928 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":316,"threads_started":1}
I20260806 01:10:41.532445 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:41.531609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:36065)
0806 01:10:41.532118 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:41.532210 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":293,"threads_started":1}
I20260806 01:10:42.029426 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:42.028640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:56211)
0806 01:10:42.029145 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:42.029241 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":264,"threads_started":1}
I20260806 01:10:42.053879 14016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:42.053017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:55461)
0806 01:10:42.053542 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:42.053635 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":239,"threads_started":1}
I20260806 01:10:42.534268 14018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:42.533384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:42153)
0806 01:10:42.533935 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:42.534026 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":234,"threads_started":1}
I20260806 01:10:43.031268 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:43.030374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:45731)
0806 01:10:43.030886 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:43.030972 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":217,"threads_started":1}
I20260806 01:10:43.055586 14021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:43.054871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:54491)
0806 01:10:43.055326 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:43.055426 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":243,"threads_started":1}
I20260806 01:10:43.536007 14022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:43.535187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:35275)
0806 01:10:43.535684 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:43.535826 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":239,"threads_started":1}
I20260806 01:10:44.033064 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:44.032278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:37623)
0806 01:10:44.032810 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:44.032909 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":287,"threads_started":1}
I20260806 01:10:44.057303 14026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:44.056445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:37273)
0806 01:10:44.056952 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:44.057103 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":246,"threads_started":1}
I20260806 01:10:44.537642 14027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:44.536946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:45489)
0806 01:10:44.537386 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:44.537467 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":217,"threads_started":1}
I20260806 01:10:45.034855 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:45.034032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:51229)
0806 01:10:45.034562 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:45.034661 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":272,"threads_started":1}
I20260806 01:10:45.059149 14030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:45.058277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:56299)
0806 01:10:45.058879 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:45.058973 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":358,"threads_started":1}
I20260806 01:10:45.539324 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:45.538616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:36389)
0806 01:10:45.539076 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:45.539159 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":256,"threads_started":1}
I20260806 01:10:46.036512 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:46.035809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:44087)
0806 01:10:46.036271 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:46.036355 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":248,"threads_started":1}
I20260806 01:10:46.060891 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:46.060150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:41959)
0806 01:10:46.060584 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:46.060684 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
I20260806 01:10:46.541049 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:46.540296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:52677)
0806 01:10:46.540772 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:46.540858 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260806 01:10:47.038204 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:47.037461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:56265)
0806 01:10:47.037925 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:47.038026 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":229,"threads_started":1}
I20260806 01:10:47.062978 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:47.062075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:46167)
0806 01:10:47.062694 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:47.062806 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":311,"threads_started":1}
I20260806 01:10:47.542936 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:47.542112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:59931)
0806 01:10:47.542605 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:47.542697 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":278,"threads_started":1}
I20260806 01:10:48.039893 14040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:48.039131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:45653)
0806 01:10:48.039587 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:48.039671 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
I20260806 01:10:48.064935 14041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:48.064175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:34317)
0806 01:10:48.064656 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:48.064748 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":241,"threads_started":1}
I20260806 01:10:48.545077 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:48.544092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:50261)
0806 01:10:48.544657 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:48.544868 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":268,"threads_started":1}
I20260806 01:10:49.041826 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:49.040923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:41897)
0806 01:10:49.041525 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:49.041633 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":313,"threads_started":1}
I20260806 01:10:49.066699 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:49.065906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:50473)
0806 01:10:49.066397 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:49.066507 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":246,"threads_started":1}
I20260806 01:10:49.547130 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:49.546251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:34937)
0806 01:10:49.546817 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:49.546925 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":243,"threads_started":1}
I20260806 01:10:50.043649 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:50.042816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:48993)
0806 01:10:50.043369 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:50.043465 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":298,"threads_started":1}
I20260806 01:10:50.068868 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:50.068009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.65:43161)
0806 01:10:50.068577 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:50.068672 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":328,"threads_started":1}
I20260806 01:10:50.549098 14052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:50.548365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.66:38543)
0806 01:10:50.548808 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:50.548907 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":232,"threads_started":1}
I20260806 01:10:51.023717 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 13325
I20260806 01:10:51.045691 14056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:51.044842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:43753 (local address 127.22.63.67:59145)
0806 01:10:51.045398 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:51.045508 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":324,"threads_started":1}
I20260806 01:10:51.046309 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:51.024912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.22.63.65:44967)
0806 01:10:51.025543 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:51.025564 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:51.025617 (+ 53us) server_negotiation.cc:407] Connection header received
0806 01:10:51.027228 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:51.027237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:51.027676 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:51.028085 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:51.029662 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.030978 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.032367 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.033039 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.033242 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:51.039907 (+ 6665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:51.039916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:51.039934 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:51.039980 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:51.042783 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.043516 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.043523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.043530 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.043705 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.044168 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.044173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.044180 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.044726 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:51.045050 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:51.045567 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 01:10:51.046043 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":363,"threads_started":1}
I20260806 01:10:51.047730 13568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:51.131404 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 13325
I20260806 01:10:51.158051 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 13580
I20260806 01:10:51.176337 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:51.159151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.22.63.66:33541)
0806 01:10:51.159684 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:51.159699 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:51.159819 (+ 120us) server_negotiation.cc:407] Connection header received
0806 01:10:51.161353 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:51.161362 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:51.161776 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:51.162074 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:51.163552 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.165013 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.166305 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.167023 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.167156 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:51.169949 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:51.169956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:51.169971 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:51.170014 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:51.173056 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.173822 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.173829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.173835 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.174011 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.174393 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.174396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.174398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.174912 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:51.175140 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:51.175582 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 01:10:51.176045 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":236,"threads_started":1}
I20260806 01:10:51.177768 13776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:51.261528 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 13580
I20260806 01:10:51.288090 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 13788
I20260806 01:10:51.305629 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:51.289176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.22.63.67:33739)
0806 01:10:51.289851 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:51.289866 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:51.289913 (+ 47us) server_negotiation.cc:407] Connection header received
0806 01:10:51.291557 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:51.291566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:51.292128 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:51.292422 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:51.293762 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.294983 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.296353 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:51.297012 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:51.297162 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:51.299886 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:51.299894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:51.299909 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:51.299943 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:51.302645 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.303301 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.303308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.303313 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.303495 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:51.303888 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:51.303892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:51.303894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:51.304417 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:51.304669 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:51.304967 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 01:10:51.305306 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":338,"threads_started":1}
I20260806 01:10:51.307168 13970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:51.393252 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 13788
2026-08-06T01:10:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25143 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:10:51 dist-test-slave-7f01 krb5kdc[14070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:10:51 dist-test-slave-7f01 krb5kdc[14070](info): set up 2 sockets
Aug 06 01:10:51 dist-test-slave-7f01 krb5kdc[14070](info): commencing operation
krb5kdc: starting...
W20260806 01:10:52.770319 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:10:52 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:10:53.079222 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:53.423842 14086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:53.424130 14086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:53.424306 14086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:53.431622 14086 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:10:53.431720 14086 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:10:53.431818 14086 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:10:53.431918 14086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:10:53.432027 14086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:53.432137 14086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:53.432219 14086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:10:53.432300 14086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:10:53.442268 14086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:53.443617 14086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:53.445032 14086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:53.457513 14102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:53.457592 14099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:53.457567 14096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:53.458370 14086 server_base.cc:1034] running on GCE node
I20260806 01:10:53.459252 14086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:53.460814 14086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:53.462095 14086 hybrid_clock.cc:648] HybridClock initialized: now 1785978653462017 us; error 36 us; skew 500 ppm
I20260806 01:10:53.462466 14086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:53 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978653, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:53.465824 14086 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:10:53.467494 14086 webserver.cc:460] Webserver started at http://127.22.63.126:37389/ using document root <none> and password file <none>
I20260806 01:10:53.468312 14086 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:53.468457 14086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:53.468835 14086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:53.472117 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "a1e450e271ff441e9d33886d4e8e7d1e"
format_stamp: "Formatted at 2026-08-06 01:10:53 on dist-test-slave-7f01"
server_key: "fc05d563ed2878c20d4521f16aba1e64"
server_key_iv: "544811c107e7f612d7bac3bb7f5341ec"
server_key_version: "encryptionkey@0"
I20260806 01:10:53.472954 14086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "a1e450e271ff441e9d33886d4e8e7d1e"
format_stamp: "Formatted at 2026-08-06 01:10:53 on dist-test-slave-7f01"
server_key: "fc05d563ed2878c20d4521f16aba1e64"
server_key_iv: "544811c107e7f612d7bac3bb7f5341ec"
server_key_version: "encryptionkey@0"
I20260806 01:10:53.478368 14086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 01:10:53.482054 14112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:53.483397 14086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:10:53.483598 14086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a1e450e271ff441e9d33886d4e8e7d1e"
format_stamp: "Formatted at 2026-08-06 01:10:53 on dist-test-slave-7f01"
server_key: "fc05d563ed2878c20d4521f16aba1e64"
server_key_iv: "544811c107e7f612d7bac3bb7f5341ec"
server_key_version: "encryptionkey@0"
I20260806 01:10:53.483937 14086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:53.505122 14086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:53.509413 14086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:53.509723 14086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:53.528951 14086 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37985
I20260806 01:10:53.528978 14189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37985 every 8 connection(s)
I20260806 01:10:53.530388 14086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:10:53.534135 14190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:53.540256 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 14086
I20260806 01:10:53.540496 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:10:53.541174 22781 external_mini_cluster.cc:1428] Setting key d62fff49c70252e8276f0bdb4090344e
I20260806 01:10:53.545138 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap starting.
I20260806 01:10:53.548947 14190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:53.550072 14190 log.cc:824] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:53.552964 14190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: No bootstrap required, opened a new log
Aug 06 01:10:53 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:53.562377 14190 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:10:53.562780 14190 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:53.562914 14190 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Initialized, Role: FOLLOWER
I20260806 01:10:53.563607 14190 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:10:53.563946 14190 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:10:53.564107 14190 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:10:53.564374 14190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:53.565485 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:53.543395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.22.63.126:37985)
0806 01:10:53.544229 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:53.544241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:53.544283 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:10:53.545994 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:53.546016 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:53.546381 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:53.546714 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:53.548726 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:53.550135 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:53.551562 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:53.552323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:53.552535 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:53.557296 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:53.557304 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:53.557341 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:53.557390 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:53.560683 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:53.561634 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:53.561640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:53.561651 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:53.561833 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:53.562313 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:53.562316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:53.562318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:53.563020 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:53.563332 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:53.563868 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 01:10:53.564358 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":381,"threads_started":1}
I20260806 01:10:53.566927 14190 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:10:53.567435 14190 leader_election.cc:302] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1e450e271ff441e9d33886d4e8e7d1e; no voters:
I20260806 01:10:53.568099 14190 leader_election.cc:288] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:10:53.568249 14197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:53.569279 14197 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 LEADER]: Becoming Leader. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Running, Role: LEADER
I20260806 01:10:53.569851 14197 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:10:53.570231 14190 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:10:53.572587 14201 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1e450e271ff441e9d33886d4e8e7d1e. Latest consensus state: current_term: 1 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:10:53.572923 14201 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:53.573558 14198 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:10:53.573849 14211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:10:53.573870 14198 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:10:53.585414 14211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:10:53.595536 14211 catalog_manager.cc:1347] Generated new cluster ID: f6b7e28941e44b928e3bcf6cf7ef14ea
I20260806 01:10:53.595672 14211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:10:53.606523 14211 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:10:53.608465 14211 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:10:53.624213 14211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Generated new TSK 0
I20260806 01:10:53.625406 14211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:10:53.701476 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:54.053054 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:54.053378 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:54.053545 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:54.060974 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:54.061126 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:54.061296 14236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:10:54.071519 14236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:54.072916 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:54.074236 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:54.087812 14250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:54.087857 14246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:54.087720 14245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:54.089316 14236 server_base.cc:1034] running on GCE node
I20260806 01:10:54.089968 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:54.090972 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:54.092238 14236 hybrid_clock.cc:648] HybridClock initialized: now 1785978654092176 us; error 60 us; skew 500 ppm
I20260806 01:10:54.092566 14236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:54 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:54.095829 14236 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:10:54.097203 14236 webserver.cc:460] Webserver started at http://127.22.63.65:37917/ using document root <none> and password file <none>
I20260806 01:10:54.097844 14236 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:54.097944 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:54.098191 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:54.101109 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "7aac8e51cd1045819df022a3200fb6d7"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "421a94554c95b45cda83b422a3fd5581"
server_key_iv: "88fc40356b5e729ef5b182273e899954"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.101852 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "7aac8e51cd1045819df022a3200fb6d7"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "421a94554c95b45cda83b422a3fd5581"
server_key_iv: "88fc40356b5e729ef5b182273e899954"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.106907 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:10:54.110309 14259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.111622 14236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:10:54.111824 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "7aac8e51cd1045819df022a3200fb6d7"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "421a94554c95b45cda83b422a3fd5581"
server_key_iv: "88fc40356b5e729ef5b182273e899954"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.112006 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:54.137760 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:54.142266 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:54.142585 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:54.143599 14236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:54.144940 14236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:54.145020 14236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.145184 14236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:54.145246 14236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.179569 14236 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44887
I20260806 01:10:54.179621 14437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44887 every 8 connection(s)
I20260806 01:10:54.180905 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:10:54.183171 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 14236
I20260806 01:10:54.183395 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:10:54.184300 22781 external_mini_cluster.cc:1428] Setting key 6830be7f66bf9e76f0a99e0889d77fab
Aug 06 01:10:54 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:54.203859 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:54.183776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:39341 (local address 127.22.63.126:37985)
0806 01:10:54.184312 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:54.184321 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:54.185302 (+ 981us) server_negotiation.cc:407] Connection header received
0806 01:10:54.187820 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:54.187823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:54.187891 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:54.188010 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:54.190103 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.190841 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.193318 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.193713 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.193809 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:54.198058 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:54.198064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:54.198067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:54.198101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:54.200768 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:54.201502 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:54.201511 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:54.201514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:54.201634 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:54.202069 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:54.202074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:54.202078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:54.202341 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:54.202501 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:54.203216 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 01:10:54.203444 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":245,"threads_started":1}
I20260806 01:10:54.204864 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:54.184283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:39341)
0806 01:10:54.185121 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:54.185162 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:54.187491 (+ 2329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:54.188268 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:54.188282 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:54.189086 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:54.189928 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:54.189955 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.190997 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.191005 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:54.193153 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:54.193178 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.193950 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.193960 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:54.194297 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:54.195055 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:54.195091 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:54.197814 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:54.200914 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:54.200926 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:54.200942 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:54.201367 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:54.201747 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:54.201752 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:54.201755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:54.201948 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:54.202481 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:54.202489 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:54.202824 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 01:10:54.203097 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 01:10:54.203633 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":307,"threads_started":1}
I20260806 01:10:54.206431 14438 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:10:54.206789 14438 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:54.207695 14438 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:10:54.209959 14132 ts_manager.cc:194] Registered new tserver with Master: 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887)
I20260806 01:10:54.211396 14132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:39341
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:10:54.254784 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:54.616134 14445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:54.616456 14445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:54.616642 14445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:54.624058 14445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:54.624172 14445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:54.624356 14445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:10:54.634654 14445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:54.636004 14445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:54.637401 14445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:54.651665 14454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:54.651635 14453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:54.651623 14460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:54.652655 14445 server_base.cc:1034] running on GCE node
I20260806 01:10:54.653219 14445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:54.654132 14445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:54.655364 14445 hybrid_clock.cc:648] HybridClock initialized: now 1785978654655316 us; error 56 us; skew 500 ppm
I20260806 01:10:54.655694 14445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:54 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:54.658825 14445 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:10:54.660383 14445 webserver.cc:460] Webserver started at http://127.22.63.66:33759/ using document root <none> and password file <none>
I20260806 01:10:54.661129 14445 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:54.661281 14445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:54.661613 14445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:54.664664 14445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "417d970652e54ef28d1a217ae7b79c3b"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "93f61bf5d44d9bd4e00bbe48378e2178"
server_key_iv: "c8360b62e227c315cb3fbaf287bdbcca"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.665434 14445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "417d970652e54ef28d1a217ae7b79c3b"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "93f61bf5d44d9bd4e00bbe48378e2178"
server_key_iv: "c8360b62e227c315cb3fbaf287bdbcca"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.670737 14445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:10:54.674427 14472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.675829 14445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:10:54.676005 14445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "417d970652e54ef28d1a217ae7b79c3b"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "93f61bf5d44d9bd4e00bbe48378e2178"
server_key_iv: "c8360b62e227c315cb3fbaf287bdbcca"
server_key_version: "encryptionkey@0"
I20260806 01:10:54.676211 14445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:54.689543 14445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:54.693547 14445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:54.693850 14445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:54.694820 14445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:54.696251 14445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:54.696355 14445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.696511 14445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:54.696597 14445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:54.731696 14445 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39585
I20260806 01:10:54.731765 14646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39585 every 8 connection(s)
I20260806 01:10:54.733019 14445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:10:54.736264 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 14445
I20260806 01:10:54.736476 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:10:54.737107 22781 external_mini_cluster.cc:1428] Setting key b9dc31dffe67b1feca2194621da40b52
Aug 06 01:10:54 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978654, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:54.756894 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:54.735602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59817 (local address 127.22.63.126:37985)
0806 01:10:54.736176 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:54.736183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:54.737212 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 01:10:54.739461 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:54.739465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:54.739529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:54.739618 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:54.741752 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.742529 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.744986 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.745661 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.745787 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:54.750435 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:54.750445 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:54.750452 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:54.750501 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:54.753739 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:54.754533 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:54.754537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:54.754540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:54.754672 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:54.755168 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:54.755171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:54.755173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:54.755423 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:54.755547 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:54.756420 (+ 873us) server_negotiation.cc:299] Negotiation successful
0806 01:10:54.756622 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":284,"threads_started":1}
I20260806 01:10:54.758435 14653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:54.736141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:59817)
0806 01:10:54.737014 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:54.737072 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:54.739173 (+ 2101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:54.739917 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:54.739927 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:54.740714 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:54.741562 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:54.741590 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.742666 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.742674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:54.744833 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:54.744851 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:54.745900 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:54.745909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:54.746242 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:54.747052 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:54.747088 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:54.750165 (+ 3077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:54.753894 (+ 3729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:54.753903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:54.753919 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:54.754384 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:54.754785 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:54.754799 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:54.754806 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:54.755048 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:54.755548 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:54.755558 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:54.756043 (+ 485us) client_negotiation.cc:769] Sending connection context
0806 01:10:54.756336 (+ 293us) client_negotiation.cc:240] Negotiation successful
0806 01:10:54.756918 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":383,"threads_started":1}
I20260806 01:10:54.759992 14647 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:10:54.760361 14647 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:54.761234 14647 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:10:54.762924 14132 ts_manager.cc:194] Registered new tserver with Master: 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585)
I20260806 01:10:54.764223 14132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:59817
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:10:54.808738 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:10:55.159322 14657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:10:55.159605 14657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:10:55.159701 14657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:10:55.167065 14657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:10:55.167184 14657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:10:55.167297 14657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:10:55.176976 14657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:10:55.178349 14657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:10:55.179883 14657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:10:55.193456 14673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:55.193549 14665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:10:55.193498 14669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:10:55.195020 14657 server_base.cc:1034] running on GCE node
I20260806 01:10:55.195690 14657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:10:55.196686 14657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:10:55.197989 14657 hybrid_clock.cc:648] HybridClock initialized: now 1785978655197935 us; error 51 us; skew 500 ppm
I20260806 01:10:55.198351 14657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:10:55 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978655, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:10:55.201819 14657 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:10:55.203333 14657 webserver.cc:460] Webserver started at http://127.22.63.67:35147/ using document root <none> and password file <none>
I20260806 01:10:55.204092 14657 fs_manager.cc:359] Metadata directory not provided
I20260806 01:10:55.204242 14657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:10:55.204562 14657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:10:55.207507 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "98b8c87134664785990de983ef880cb5"
format_stamp: "Formatted at 2026-08-06 01:10:55 on dist-test-slave-7f01"
server_key: "bdd022764d1f32d912af07b204e30044"
server_key_iv: "9a3941b1de2f5258bcc5ca6af2cd6cce"
server_key_version: "encryptionkey@0"
I20260806 01:10:55.208348 14657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "98b8c87134664785990de983ef880cb5"
format_stamp: "Formatted at 2026-08-06 01:10:55 on dist-test-slave-7f01"
server_key: "bdd022764d1f32d912af07b204e30044"
server_key_iv: "9a3941b1de2f5258bcc5ca6af2cd6cce"
server_key_version: "encryptionkey@0"
I20260806 01:10:55.213370 14657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:10:55.215129 14438 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:10:55.217298 14682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:55.218736 14657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:10:55.218935 14657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "98b8c87134664785990de983ef880cb5"
format_stamp: "Formatted at 2026-08-06 01:10:55 on dist-test-slave-7f01"
server_key: "bdd022764d1f32d912af07b204e30044"
server_key_iv: "9a3941b1de2f5258bcc5ca6af2cd6cce"
server_key_version: "encryptionkey@0"
I20260806 01:10:55.219167 14657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:10:55.244999 14657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:10:55.249814 14657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:10:55.250147 14657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:10:55.251161 14657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:10:55.252666 14657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:10:55.252775 14657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:55.252928 14657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:10:55.253012 14657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:10:55.288832 14657 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:46545
I20260806 01:10:55.288893 14850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:46545 every 8 connection(s)
I20260806 01:10:55.290190 14657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:10:55.299600 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 14657
I20260806 01:10:55.299860 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:10:55.300324 22781 external_mini_cluster.cc:1428] Setting key 97fa085c673518f338852d982ec92a6e
Aug 06 01:10:55 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978655, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:10:55.310715 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.292904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:58075 (local address 127.22.63.126:37985)
0806 01:10:55.293461 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.293467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.294237 (+ 770us) server_negotiation.cc:407] Connection header received
0806 01:10:55.296196 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.296202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.296296 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.296425 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:55.297922 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.298763 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.300560 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.301095 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.301250 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.304874 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:55.304879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:55.304883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:55.304917 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:55.307504 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.308302 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.308307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.308309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.308432 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.308892 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.308896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.308898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.309172 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:55.309345 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.310102 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.310330 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":294,"threads_started":1}
I20260806 01:10:55.311673 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.293362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:58075)
0806 01:10:55.294075 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.294112 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.295926 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.296656 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.296665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.297177 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:10:55.297745 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.297761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.298917 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.298922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.300337 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.300350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.301362 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.301373 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.301627 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.302371 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:10:55.302394 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:10:55.304655 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:10:55.307647 (+ 2992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:55.307657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:55.307673 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:55.308164 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:55.308551 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:10:55.308556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:10:55.308560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:10:55.308758 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:10:55.309318 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:10:55.309326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:10:55.309705 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.309981 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.310536 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":275,"threads_started":1}
I20260806 01:10:55.313125 14851 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:10:55.313457 14851 heartbeater.cc:464] Registering TS with master...
I20260806 01:10:55.314222 14851 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:10:55.315815 14131 ts_manager.cc:194] Registered new tserver with Master: 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545)
I20260806 01:10:55.316744 14131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:58075
I20260806 01:10:55.328024 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:10:55.345603 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.332748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.22.63.126:37985)
0806 01:10:55.332981 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.332989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.333381 (+ 392us) server_negotiation.cc:407] Connection header received
0806 01:10:55.333512 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.333516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.333583 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.333664 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:55.335033 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.335811 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.337289 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.337679 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.337784 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.340315 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:55.340322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:55.340327 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:55.340366 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:55.343095 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.343764 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.343790 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.343793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.343916 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.344298 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.344301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.344303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.344613 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:55.344831 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.345094 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.345256 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:10:55.349920 14131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:10:55.357276 14131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:10:55.388306 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.369535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46545 (local address 127.0.0.1:57950)
0806 01:10:55.381280 (+ 11745us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.381328 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.381528 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.382217 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.382224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.382256 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:55.382681 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.382692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.384250 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.384258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.385809 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.385822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.387149 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.387159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.387378 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.387450 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.387633 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.387876 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11383,"thread_start_us":11235,"threads_started":1}
I20260806 01:10:55.388897 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.369899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.22.63.67:46545)
0806 01:10:55.380562 (+ 10663us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.380573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.381497 (+ 924us) server_negotiation.cc:407] Connection header received
0806 01:10:55.381779 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.381790 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.382007 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.382207 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:55.382957 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.384116 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.385994 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.386918 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.387076 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.387358 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.388348 (+ 990us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.388539 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10435,"thread_start_us":10225,"threads_started":1}
I20260806 01:10:55.393224 14753 tablet_service.cc:1467] Processing CreateTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 (DEFAULT_TABLE table=test-table [id=0a960f13c3d84ca8864306dd71062993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:55.394855 14753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49928b73eafa40649c1ec2c4f387e4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:55.396842 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.371525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:56770)
0806 01:10:55.387903 (+ 16378us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.387929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.388072 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.389184 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.389187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.389209 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:55.389534 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.389542 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.391646 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.391654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.393581 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.393594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.395904 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.395912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.396125 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.396162 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.396251 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.396396 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16230,"mutex_wait_us":73,"thread_start_us":16026,"threads_started":1}
I20260806 01:10:55.397327 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.372103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39585 (local address 127.0.0.1:36298)
0806 01:10:55.388403 (+ 16300us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.388426 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.388545 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.389806 (+ 1261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.389810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.389835 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:55.390214 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.390227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.392208 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.392213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.394333 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.394352 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.396521 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.396530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.396707 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.396737 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.396854 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.397000 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16107,"thread_start_us":15851,"threads_started":1}
I20260806 01:10:55.397670 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.371437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.22.63.65:44887)
0806 01:10:55.386892 (+ 15455us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.386904 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.388228 (+ 1324us) server_negotiation.cc:407] Connection header received
0806 01:10:55.388347 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.388358 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.388947 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.389178 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:55.389709 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.391509 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.393749 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.395329 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.395471 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.395712 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.397145 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.397326 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15191,"thread_start_us":16078,"threads_started":1}
I20260806 01:10:55.397940 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.370198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.22.63.66:39585)
0806 01:10:55.379993 (+ 9795us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.380006 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.388663 (+ 8657us) server_negotiation.cc:407] Connection header received
0806 01:10:55.388768 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.388775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.388972 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.389828 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:55.390410 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.392024 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.394534 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.396022 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.396160 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.396362 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.397160 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.397339 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9593,"thread_start_us":9473,"threads_started":1}
I20260806 01:10:55.401536 14324 tablet_service.cc:1467] Processing CreateTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 (DEFAULT_TABLE table=test-table [id=0a960f13c3d84ca8864306dd71062993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:55.401536 14544 tablet_service.cc:1467] Processing CreateTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 (DEFAULT_TABLE table=test-table [id=0a960f13c3d84ca8864306dd71062993]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:10:55.403075 14324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49928b73eafa40649c1ec2c4f387e4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:55.403082 14544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49928b73eafa40649c1ec2c4f387e4f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:10:55.408162 14885 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Bootstrap starting.
I20260806 01:10:55.411916 14885 tablet_bootstrap.cc:654] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:55.412992 14885 log.cc:824] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:55.416060 14885 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: No bootstrap required, opened a new log
I20260806 01:10:55.416353 14885 ts_tablet_manager.cc:1397] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260806 01:10:55.419831 14888 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Bootstrap starting.
I20260806 01:10:55.420634 14887 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Bootstrap starting.
I20260806 01:10:55.424036 14888 tablet_bootstrap.cc:654] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:55.424212 14887 tablet_bootstrap.cc:654] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Neither blocks nor log segments found. Creating new log.
I20260806 01:10:55.425179 14888 log.cc:824] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:55.425411 14887 log.cc:824] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:10:55.427358 14885 raft_consensus.cc:348] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.427747 14885 raft_consensus.cc:374] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:55.427871 14885 raft_consensus.cc:729] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b8c87134664785990de983ef880cb5, State: Initialized, Role: FOLLOWER
I20260806 01:10:55.428678 14888 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: No bootstrap required, opened a new log
I20260806 01:10:55.428550 14885 consensus_queue.cc:260] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.428997 14888 ts_tablet_manager.cc:1397] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260806 01:10:55.429844 14851 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:10:55.430217 14887 tablet_bootstrap.cc:492] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: No bootstrap required, opened a new log
I20260806 01:10:55.430526 14885 ts_tablet_manager.cc:1428] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260806 01:10:55.430563 14887 ts_tablet_manager.cc:1397] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 01:10:55.438170 14888 raft_consensus.cc:348] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.438628 14888 raft_consensus.cc:374] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:55.438741 14888 raft_consensus.cc:729] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417d970652e54ef28d1a217ae7b79c3b, State: Initialized, Role: FOLLOWER
I20260806 01:10:55.439306 14888 consensus_queue.cc:260] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.439785 14887 raft_consensus.cc:348] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.440140 14887 raft_consensus.cc:374] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:10:55.440266 14887 raft_consensus.cc:729] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aac8e51cd1045819df022a3200fb6d7, State: Initialized, Role: FOLLOWER
I20260806 01:10:55.440567 14647 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:10:55.440883 14887 consensus_queue.cc:260] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.441140 14888 ts_tablet_manager.cc:1428] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260806 01:10:55.442785 14887 ts_tablet_manager.cc:1428] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Time spent starting tablet: real 0.012s user 0.002s sys 0.010s
W20260806 01:10:55.485126 14648 tablet.cc:2369] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:10:55.541934 14853 tablet.cc:2369] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:55.665997 14899 raft_consensus.cc:482] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:10:55.666357 14899 raft_consensus.cc:504] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.668200 14899 leader_election.cc:288] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545), 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585)
I20260806 01:10:55.675663 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.668698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:34509 (local address 127.22.63.67:46545)
0806 01:10:55.668970 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.668979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.669868 (+ 889us) server_negotiation.cc:407] Connection header received
0806 01:10:55.670101 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.670108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.670193 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.670297 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:55.670911 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.671941 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.673869 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.674643 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.674837 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.674923 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.675247 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.675372 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:10:55.675866 14900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.668686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46545 (local address 127.22.63.65:34509)
0806 01:10:55.669730 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.669755 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.669895 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.670322 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.670326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.670351 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:55.670757 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.670768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.672068 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.672075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.673703 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.673716 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.674875 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.674882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.675056 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.675106 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.675215 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.675348 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":19,"thread_start_us":755,"threads_started":1}
I20260806 01:10:55.676013 14901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.668705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39585 (local address 127.22.63.65:41999)
0806 01:10:55.669846 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0806 01:10:55.669865 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:10:55.670012 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:10:55.670421 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:10:55.670426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:10:55.670467 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:10:55.670907 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.670916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.672077 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.672082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.673708 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:10:55.673721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.674928 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.674934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:10:55.675113 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.675136 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 01:10:55.675243 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 01:10:55.675371 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"mutex_wait_us":102,"thread_start_us":494,"threads_started":1}
I20260806 01:10:55.675688 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.668792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:41999 (local address 127.22.63.66:39585)
0806 01:10:55.669064 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.669073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.669953 (+ 880us) server_negotiation.cc:407] Connection header received
0806 01:10:55.670157 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.670163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.670294 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.670403 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:10:55.671068 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.671920 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.673878 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.674668 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.674858 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.674943 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.675279 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.675413 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:10:55.677203 14576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49928b73eafa40649c1ec2c4f387e4f4" candidate_uuid: "7aac8e51cd1045819df022a3200fb6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417d970652e54ef28d1a217ae7b79c3b" is_pre_election: true
I20260806 01:10:55.677217 14787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49928b73eafa40649c1ec2c4f387e4f4" candidate_uuid: "7aac8e51cd1045819df022a3200fb6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b8c87134664785990de983ef880cb5" is_pre_election: true
I20260806 01:10:55.677834 14787 raft_consensus.cc:2454] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aac8e51cd1045819df022a3200fb6d7 in term 0.
I20260806 01:10:55.677850 14576 raft_consensus.cc:2454] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aac8e51cd1045819df022a3200fb6d7 in term 0.
I20260806 01:10:55.678588 14262 leader_election.cc:302] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417d970652e54ef28d1a217ae7b79c3b, 7aac8e51cd1045819df022a3200fb6d7; no voters:
I20260806 01:10:55.679015 14899 raft_consensus.cc:2781] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:10:55.679177 14899 raft_consensus.cc:482] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:10:55.679339 14899 raft_consensus.cc:3037] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:55.681936 14899 raft_consensus.cc:504] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.682691 14899 leader_election.cc:288] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [CANDIDATE]: Term 1 election: Requested vote from peers 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545), 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585)
I20260806 01:10:55.683195 14576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49928b73eafa40649c1ec2c4f387e4f4" candidate_uuid: "7aac8e51cd1045819df022a3200fb6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "417d970652e54ef28d1a217ae7b79c3b"
I20260806 01:10:55.683207 14787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49928b73eafa40649c1ec2c4f387e4f4" candidate_uuid: "7aac8e51cd1045819df022a3200fb6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b8c87134664785990de983ef880cb5"
I20260806 01:10:55.683439 14576 raft_consensus.cc:3037] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:55.683427 14787 raft_consensus.cc:3037] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:10:55.685814 14787 raft_consensus.cc:2454] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aac8e51cd1045819df022a3200fb6d7 in term 1.
I20260806 01:10:55.686069 14576 raft_consensus.cc:2454] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aac8e51cd1045819df022a3200fb6d7 in term 1.
I20260806 01:10:55.686314 14265 leader_election.cc:302] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aac8e51cd1045819df022a3200fb6d7, 98b8c87134664785990de983ef880cb5; no voters:
W20260806 01:10:55.686375 14439 tablet.cc:2369] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:10:55.686707 14899 raft_consensus.cc:2781] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:10:55.687350 14899 raft_consensus.cc:686] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 7aac8e51cd1045819df022a3200fb6d7, State: Running, Role: LEADER
I20260806 01:10:55.687932 14899 consensus_queue.cc:237] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } }
I20260806 01:10:55.692337 14131 catalog_manager.cc:5495] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "7aac8e51cd1045819df022a3200fb6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } health_report { overall_health: HEALTHY } } }
I20260806 01:10:55.766604 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.761709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.22.63.65:44887)
0806 01:10:55.761904 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.761908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.761925 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:10:55.762015 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.762018 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.762095 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.762216 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:10:55.762552 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.763261 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.764702 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.765118 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.765221 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.765638 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:10:55.766134 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:10:55.766247 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.766335 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.766401 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:10:55.774887 14787 raft_consensus.cc:1261] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Refusing update from remote peer 7aac8e51cd1045819df022a3200fb6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:55.774885 14576 raft_consensus.cc:1261] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Refusing update from remote peer 7aac8e51cd1045819df022a3200fb6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:10:55.775899 14904 consensus_queue.cc:1035] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:55.776237 14899 consensus_queue.cc:1035] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:10:55.792385 14911 mvcc.cc:205] Tried to move back new op lower bound from 7315368574044327936 to 7315368573701365760. Current Snapshot: MvccSnapshot[applied={T|T < 7315368574044327936}]
I20260806 01:10:55.804289 14132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:55.804561 14132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:10:55.808400 14132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Sending DeleteTablet for 3 replicas of tablet 49928b73eafa40649c1ec2c4f387e4f4
I20260806 01:10:55.809597 14753 tablet_service.cc:1514] Processing DeleteTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:55 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:57950
I20260806 01:10:55.809830 14324 tablet_service.cc:1514] Processing DeleteTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:55 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:56770
I20260806 01:10:55.809839 14544 tablet_service.cc:1514] Processing DeleteTablet for tablet 49928b73eafa40649c1ec2c4f387e4f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:10:55 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260806 01:10:55.810883 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 14236
I20260806 01:10:55.820770 14933 tablet_replica.cc:330] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: stopping tablet replica
I20260806 01:10:55.820922 14927 tablet_replica.cc:330] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: stopping tablet replica
I20260806 01:10:55.821295 14927 raft_consensus.cc:2229] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:55.821285 14933 raft_consensus.cc:2229] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:10:55.821822 14933 raft_consensus.cc:2258] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:55.821818 14927 raft_consensus.cc:2258] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:55.824270 14930 tablet_replica.cc:330] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: stopping tablet replica
I20260806 01:10:55.824573 14927 ts_tablet_manager.cc:1905] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:55.824692 14930 raft_consensus.cc:2229] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:10:55.825388 14930 raft_consensus.cc:2258] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:10:55.827539 14933 ts_tablet_manager.cc:1905] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:55.831334 14930 ts_tablet_manager.cc:1905] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:10:55.831905 14927 ts_tablet_manager.cc:1918] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:55.832084 14927 log.cc:1196] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/49928b73eafa40649c1ec2c4f387e4f4
I20260806 01:10:55.832607 14927 ts_tablet_manager.cc:1939] T 49928b73eafa40649c1ec2c4f387e4f4 P 98b8c87134664785990de983ef880cb5: Deleting consensus metadata
I20260806 01:10:55.834511 14115 catalog_manager.cc:4841] TS 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545): tablet 49928b73eafa40649c1ec2c4f387e4f4 (table test-table [id=0a960f13c3d84ca8864306dd71062993]) successfully deleted
I20260806 01:10:55.838928 14930 ts_tablet_manager.cc:1918] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 06 01:10:55 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:10:55.839109 14930 log.cc:1196] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/49928b73eafa40649c1ec2c4f387e4f4
I20260806 01:10:55.839694 14930 ts_tablet_manager.cc:1939] T 49928b73eafa40649c1ec2c4f387e4f4 P 7aac8e51cd1045819df022a3200fb6d7: Deleting consensus metadata
I20260806 01:10:55.840498 14933 ts_tablet_manager.cc:1918] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:10:55.840673 14933 log.cc:1196] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/49928b73eafa40649c1ec2c4f387e4f4
I20260806 01:10:55.841214 14933 ts_tablet_manager.cc:1939] T 49928b73eafa40649c1ec2c4f387e4f4 P 417d970652e54ef28d1a217ae7b79c3b: Deleting consensus metadata
I20260806 01:10:55.841645 14113 catalog_manager.cc:4841] TS 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887): tablet 49928b73eafa40649c1ec2c4f387e4f4 (table test-table [id=0a960f13c3d84ca8864306dd71062993]) successfully deleted
I20260806 01:10:55.842846 14114 catalog_manager.cc:4841] TS 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585): tablet 49928b73eafa40649c1ec2c4f387e4f4 (table test-table [id=0a960f13c3d84ca8864306dd71062993]) successfully deleted
I20260806 01:10:55.848265 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.812049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.22.63.65:44887)
0806 01:10:55.812227 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.812234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.826824 (+ 14590us) server_negotiation.cc:407] Connection header received
0806 01:10:55.827316 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.827322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.827417 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.827502 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:55.829836 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.831061 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.833514 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.834278 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.834378 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.840584 (+ 6206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:55.840600 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:55.840619 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:55.840672 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:55.844990 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.845767 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.845776 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.845781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.845979 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.846499 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.846505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.846508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.846979 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:55.847298 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.847617 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.847886 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:10:55.849048 14428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:55.953701 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 14236
I20260806 01:10:55.982580 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 14445
Aug 06 01:10:55 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:10:55.998494 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:55.983422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.22.63.66:39585)
0806 01:10:55.983592 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:55.983597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:55.983616 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:10:55.983848 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:55.983853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:55.983937 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:55.984023 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:55.985342 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.986095 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.987669 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:55.988273 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:55.988378 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:55.992903 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:55.992918 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:55.992935 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:55.992974 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:55.995604 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.996313 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.996321 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.996323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.996467 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:55.996851 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:55.996858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:55.996865 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:55.997248 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:55.997555 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:55.997908 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 01:10:55.998137 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:10:55.999233 14641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:56.089155 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 14445
I20260806 01:10:56.115868 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 14657
Aug 06 01:10:56 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:10:56.134522 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:10:56.116745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.22.63.67:46545)
0806 01:10:56.116964 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:10:56.116968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:10:56.116990 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:10:56.117096 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:10:56.117100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:10:56.117181 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:10:56.117285 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:10:56.118686 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:56.119403 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:56.120895 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:10:56.121309 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:10:56.121417 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:10:56.128947 (+ 7530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:10:56.128965 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:10:56.128982 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:10:56.129019 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:10:56.131974 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:56.132676 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:56.132684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:56.132686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:56.132852 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:10:56.133229 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:10:56.133234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:10:56.133236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:10:56.133615 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:10:56.133880 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 01:10:56.134087 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 01:10:56.134248 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:10:56.135280 14844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:56.224587 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 14657
I20260806 01:10:56.249024 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 14086
I20260806 01:10:56.250165 14179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:10:56.361630 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 14086
I20260806 01:11:16.392009 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--webserver_interface=127.22.63.126
--webserver_port=37389
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:11:16.741571 14940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:16.741995 14940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:16.742157 14940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:16.749536 14940 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:11:16.749670 14940 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:11:16.749764 14940 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:11:16.749871 14940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:11:16.749974 14940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:16.750098 14940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:16.750187 14940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:11:16.750268 14940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:11:16.760210 14940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=37389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:16.761528 14940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:16.763007 14940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:16.788856 14951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:16.788817 14949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:16.788878 14948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:16.789758 14940 server_base.cc:1034] running on GCE node
I20260806 01:11:16.790786 14940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:16.792622 14940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:16.793888 14940 hybrid_clock.cc:648] HybridClock initialized: now 1785978676793811 us; error 51 us; skew 500 ppm
I20260806 01:11:16.794301 14940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:16 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978676, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:16.798110 14940 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:11:16.799796 14940 webserver.cc:460] Webserver started at http://127.22.63.126:37389/ using document root <none> and password file <none>
I20260806 01:11:16.800578 14940 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:16.800688 14940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:16.806836 14940 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 01:11:16.809840 14960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:16.811223 14940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:11:16.811410 14940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a1e450e271ff441e9d33886d4e8e7d1e"
format_stamp: "Formatted at 2026-08-06 01:10:53 on dist-test-slave-7f01"
server_key: "fc05d563ed2878c20d4521f16aba1e64"
server_key_iv: "544811c107e7f612d7bac3bb7f5341ec"
server_key_version: "encryptionkey@0"
I20260806 01:11:16.812536 14940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:16.830538 14940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:16.835445 14940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:16.835808 14940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:16.855373 14940 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37985
I20260806 01:11:16.855398 15031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37985 every 8 connection(s)
I20260806 01:11:16.857353 14940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:11:16.864449 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 14940
I20260806 01:11:16.865232 15032 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:11:16.865593 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:44887
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=37917
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:11:16.870723 15032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap starting.
I20260806 01:11:16.893656 15032 log.cc:824] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:16.906594 15032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:11:16.907222 15032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap complete.
I20260806 01:11:16.917234 15032 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:16.918078 15032 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Initialized, Role: FOLLOWER
I20260806 01:11:16.918708 15032 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:16.918970 15032 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:11:16.919099 15032 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:11:16.919291 15032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 1 FOLLOWER]: Advancing to term 2
I20260806 01:11:16.923192 15032 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:16.923837 15032 leader_election.cc:302] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1e450e271ff441e9d33886d4e8e7d1e; no voters:
I20260806 01:11:16.924504 15032 leader_election.cc:288] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 01:11:16.924655 15036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 01:11:16.925922 15036 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 LEADER]: Becoming Leader. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Running, Role: LEADER
I20260806 01:11:16.926506 15036 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:16.926899 15032 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:11:16.929392 15041 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1e450e271ff441e9d33886d4e8e7d1e. Latest consensus state: current_term: 2 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:11:16.929759 15041 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:16.930346 15038 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:11:16.930701 15038 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:16.941352 15046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:11:16.949920 15046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:11:16.951407 15046 catalog_manager.cc:1259] Loaded cluster ID: f6b7e28941e44b928e3bcf6cf7ef14ea
I20260806 01:11:16.951561 15046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:11:16.955716 15046 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:11:16.957353 15046 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Loaded TSK: 0
I20260806 01:11:16.992344 15046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Generated new TSK 1
I20260806 01:11:16.992777 15046 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 01:11:17.247032 15034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:17.247360 15034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:17.247530 15034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:17.254757 15034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:17.254915 15034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:17.255091 15034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:11:17.264721 15034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=37917
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:17.266104 15034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:17.267518 15034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:17.282855 15072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:17.282837 15069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:17.283601 15034 server_base.cc:1034] running on GCE node
W20260806 01:11:17.283052 15076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:17.284492 15034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:17.285646 15034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:17.286902 15034 hybrid_clock.cc:648] HybridClock initialized: now 1785978677286848 us; error 44 us; skew 500 ppm
I20260806 01:11:17.287286 15034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:17 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978677, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:17.291569 15034 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:11:17.293411 15034 webserver.cc:460] Webserver started at http://127.22.63.65:37917/ using document root <none> and password file <none>
I20260806 01:11:17.294257 15034 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:17.294404 15034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:17.301422 15034 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 01:11:17.304700 15083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:17.306313 15034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 01:11:17.306483 15034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "7aac8e51cd1045819df022a3200fb6d7"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "421a94554c95b45cda83b422a3fd5581"
server_key_iv: "88fc40356b5e729ef5b182273e899954"
server_key_version: "encryptionkey@0"
I20260806 01:11:17.307861 15034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:17.337533 15034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:17.342265 15034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:17.342603 15034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:17.343917 15034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:17.346290 15034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:17.346463 15034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 01:11:17.346601 15034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:17.346700 15034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:17.383723 15034 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44887
I20260806 01:11:17.383875 15262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44887 every 8 connection(s)
I20260806 01:11:17.385303 15034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:11:17.391191 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 15034
I20260806 01:11:17.392028 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:39585
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=33759
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:11:17 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978677, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:11:17.411391 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:17.388321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:34161 (local address 127.22.63.126:37985)
0806 01:11:17.389020 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:17.389042 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:17.389669 (+ 627us) server_negotiation.cc:407] Connection header received
0806 01:11:17.391986 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:17.392007 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:17.392451 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:17.392838 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:17.394208 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.396221 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.397735 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.398788 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.399012 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:17.402768 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:17.402780 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:17.402801 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:17.402869 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:17.406433 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:17.407506 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:17.407513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:17.407519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:17.407689 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:17.408358 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:17.408362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:17.408364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:17.408878 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:17.409034 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:17.410077 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0806 01:11:17.410417 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":287,"threads_started":1}
I20260806 01:11:17.411868 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:17.388524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:34161)
0806 01:11:17.389484 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:17.389530 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:17.391302 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:17.392954 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:17.392961 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:17.393463 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:11:17.393986 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:17.394003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.396393 (+ 2390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.396397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:17.397584 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:17.397598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.399028 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.399033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:17.399222 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:17.399992 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:11:17.400016 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:11:17.402474 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:11:17.406603 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:17.406615 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:17.406634 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:17.407344 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:17.407947 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:17.407955 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:17.407962 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:17.408215 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:17.409038 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:11:17.409049 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:11:17.409539 (+ 490us) client_negotiation.cc:769] Sending connection context
0806 01:11:17.409898 (+ 359us) client_negotiation.cc:240] Negotiation successful
0806 01:11:17.410497 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":479,"threads_started":1}
I20260806 01:11:17.414888 15263 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:11:17.415242 15263 heartbeater.cc:464] Registering TS with master...
I20260806 01:11:17.416232 15263 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:11:17.418991 14984 ts_manager.cc:194] Registered new tserver with Master: 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887)
I20260806 01:11:17.421330 14984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:34161
W20260806 01:11:17.747968 15269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:17.748274 15269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:17.748373 15269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:17.755576 15269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:17.755695 15269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:17.755882 15269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:11:17.765594 15269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:39585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=33759
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:17.766929 15269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:17.768276 15269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:17.781754 15281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:17.781862 15278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:17.781824 15277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:17.782433 15269 server_base.cc:1034] running on GCE node
I20260806 01:11:17.783212 15269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:17.784324 15269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:17.785542 15269 hybrid_clock.cc:648] HybridClock initialized: now 1785978677785502 us; error 38 us; skew 500 ppm
I20260806 01:11:17.785907 15269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:17 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978677, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:17.789256 15269 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:11:17.790791 15269 webserver.cc:460] Webserver started at http://127.22.63.66:33759/ using document root <none> and password file <none>
I20260806 01:11:17.791532 15269 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:17.791700 15269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:17.797312 15269 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:11:17.800164 15290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:17.801564 15269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 01:11:17.801753 15269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "417d970652e54ef28d1a217ae7b79c3b"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "93f61bf5d44d9bd4e00bbe48378e2178"
server_key_iv: "c8360b62e227c315cb3fbaf287bdbcca"
server_key_version: "encryptionkey@0"
I20260806 01:11:17.802786 15269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:17.819227 15269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:17.823853 15269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:17.824149 15269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:17.825168 15269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:17.826582 15269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:17.826711 15269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:17.826869 15269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:17.826967 15269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:17.863252 15269 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39585
I20260806 01:11:17.863356 15443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39585 every 8 connection(s)
I20260806 01:11:17.864907 15269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:11:17.869686 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 15269
I20260806 01:11:17.870590 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:46545
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=35147
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 01:11:17 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978677, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:11:17.891674 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:17.867819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:36267 (local address 127.22.63.126:37985)
0806 01:11:17.868057 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:17.868064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:17.869282 (+ 1218us) server_negotiation.cc:407] Connection header received
0806 01:11:17.871990 (+ 2708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:17.871999 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:17.872135 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:17.872261 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:17.874234 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.875353 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.878189 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.878989 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.879146 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:17.884052 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:17.884061 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:17.884069 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:17.884119 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:17.888092 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:17.888906 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:17.888915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:17.888921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:17.889082 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:17.889541 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:17.889549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:17.889555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:17.889947 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:17.890100 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:17.890930 (+ 830us) server_negotiation.cc:299] Negotiation successful
0806 01:11:17.891201 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:11:17.892407 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:17.868302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:36267)
0806 01:11:17.869107 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:17.869172 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:17.871629 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:17.872536 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:17.872547 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:17.873277 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:11:17.874018 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:17.874040 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.875546 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.875550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:17.877996 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:17.878011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:17.879317 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:17.879326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:17.879740 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:17.880686 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:11:17.880720 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:11:17.883775 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:11:17.888279 (+ 4504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:17.888296 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:17.888310 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:17.888751 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:17.889194 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:17.889197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:17.889200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:17.889399 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:17.890186 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:11:17.890193 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:11:17.890543 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 01:11:17.890804 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 01:11:17.891221 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260806 01:11:17.894145 15444 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:11:17.894496 15444 heartbeater.cc:464] Registering TS with master...
I20260806 01:11:17.895278 15444 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:11:17.896945 14984 ts_manager.cc:194] Registered new tserver with Master: 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585)
I20260806 01:11:17.897745 14984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:36267
W20260806 01:11:18.226651 15447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:18.227010 15447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:18.227181 15447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:18.234589 15447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:18.234753 15447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:18.234939 15447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:11:18.245044 15447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:46545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=35147
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:18.246423 15447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:18.247879 15447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:18.261247 15456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:18.261303 15459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:18.261307 15455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:18.262076 15447 server_base.cc:1034] running on GCE node
I20260806 01:11:18.262730 15447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:18.263811 15447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:18.265066 15447 hybrid_clock.cc:648] HybridClock initialized: now 1785978678265032 us; error 36 us; skew 500 ppm
I20260806 01:11:18.265415 15447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:18 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:18.268693 15447 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:11:18.270294 15447 webserver.cc:460] Webserver started at http://127.22.63.67:35147/ using document root <none> and password file <none>
I20260806 01:11:18.271015 15447 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:18.271183 15447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:18.276698 15447 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:11:18.279498 15476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:18.280946 15447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:11:18.281137 15447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "98b8c87134664785990de983ef880cb5"
format_stamp: "Formatted at 2026-08-06 01:10:55 on dist-test-slave-7f01"
server_key: "bdd022764d1f32d912af07b204e30044"
server_key_iv: "9a3941b1de2f5258bcc5ca6af2cd6cce"
server_key_version: "encryptionkey@0"
I20260806 01:11:18.282153 15447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:18.295449 15447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:18.300033 15447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:18.300315 15447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:18.301309 15447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:18.302690 15447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:18.302827 15447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:18.302953 15447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:18.303045 15447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:18.338361 15447 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:46545
I20260806 01:11:18.338455 15651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:46545 every 8 connection(s)
I20260806 01:11:18.339951 15447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:11:18.347829 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 15447
Aug 06 01:11:18 dist-test-slave-7f01 krb5kdc[14070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978678, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:11:18.361902 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.342621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:49001 (local address 127.22.63.126:37985)
0806 01:11:18.342848 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.342854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.344049 (+ 1195us) server_negotiation.cc:407] Connection header received
0806 01:11:18.345853 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.345858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.345929 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.346026 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:18.347569 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.348521 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.350546 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.350980 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.351114 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.355079 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:18.355086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:18.355091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:18.355134 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:18.358581 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.359505 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.359509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.359511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.359636 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.360159 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.360167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.360171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.360453 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:18.360629 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.361312 (+ 683us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.361527 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:11:18.362988 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.343104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:49001)
0806 01:11:18.343882 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.343928 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.345586 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.346241 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.346249 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.346761 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:11:18.347365 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.347384 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.348713 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.348717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.350391 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.350408 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.351215 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.351223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.351462 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.352258 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:11:18.352286 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:11:18.354833 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:11:18.358718 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:18.358730 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:18.358747 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:18.359378 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:18.359768 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:18.359773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:18.359776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:18.360037 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:18.360599 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:11:18.360607 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:11:18.360946 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.361210 (+ 264us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.361778 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":272,"threads_started":1}
I20260806 01:11:18.364069 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.348793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.22.63.126:37985)
0806 01:11:18.349285 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.349292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.349717 (+ 425us) server_negotiation.cc:407] Connection header received
0806 01:11:18.349843 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.349848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.349924 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.350033 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:18.351560 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.352623 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.354031 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.354678 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.354804 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.358366 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:18.358371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:18.358374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:18.358424 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:18.361298 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.361961 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.361970 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.361976 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.362135 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.362511 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.362516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.362519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.362772 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:18.363036 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.363390 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.363619 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"spinlock_wait_cycles":35584,"thread_start_us":257,"threads_started":1}
I20260806 01:11:18.364697 15653 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37985
I20260806 01:11:18.365026 15653 heartbeater.cc:464] Registering TS with master...
I20260806 01:11:18.365837 15653 heartbeater.cc:511] Master 127.22.63.126:37985 requested a full tablet report, sending...
I20260806 01:11:18.367317 14984 ts_manager.cc:194] Registered new tserver with Master: 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545)
I20260806 01:11:18.368171 14984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:49001
I20260806 01:11:18.379204 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:11:18.387013 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.380182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.22.63.126:37985)
0806 01:11:18.380411 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.380419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.380895 (+ 476us) server_negotiation.cc:407] Connection header received
0806 01:11:18.381098 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.381102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.381243 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.381369 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:11:18.381815 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.382782 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.384641 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.385291 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.385431 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.386019 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:11:18.386296 (+ 277us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 01:11:18.386667 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47694: token expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 01:11:18.387156 15657 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47694: token expired
W20260806 01:11:18.387696 14858 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.63.126:37985)
I20260806 01:11:18.400485 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.388338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.22.63.126:37985)
0806 01:11:18.388497 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.388502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.388522 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:11:18.388670 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.388675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.388746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.388840 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:18.390162 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.390931 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.392440 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.392824 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.392968 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.395665 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:18.395670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:18.395673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:18.395705 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:18.398137 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.398769 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.398773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.398775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.398894 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.399219 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.399223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.399224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.399529 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:18.399702 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.400078 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.400247 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:11:18.403858 14984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:11:18.411216 14984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:11:18.425323 15263 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:11:18.450716 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.423064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.22.63.66:39585)
0806 01:11:18.442206 (+ 19142us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.442218 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.442257 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:11:18.442407 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.442419 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.442712 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.442940 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:11:18.443957 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.445321 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.448445 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.449529 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.449687 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.449900 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.450203 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.450366 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18884,"thread_start_us":19164,"threads_started":1}
I20260806 01:11:18.450757 15668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.422819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:39585 (local address 127.0.0.1:39196)
0806 01:11:18.438026 (+ 15207us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.438087 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.438284 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.442960 (+ 4676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.442969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.443001 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:11:18.443706 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.443722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.445449 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.445457 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.448267 (+ 2810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.448282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.449758 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.449765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.449966 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.450009 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.450135 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.450329 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14896,"mutex_wait_us":2,"thread_start_us":15669,"threads_started":1}
I20260806 01:11:18.451854 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.423187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35922)
0806 01:11:18.442922 (+ 19735us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.442947 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.443227 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.444204 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.444210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.444233 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:11:18.444700 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.444713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.446714 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.446719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.448625 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.448640 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.450946 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.450956 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.451144 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.451177 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.451265 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.451392 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19557,"mutex_wait_us":28,"thread_start_us":20112,"threads_started":1}
I20260806 01:11:18.452034 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.424020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.22.63.67:46545)
0806 01:11:18.441574 (+ 17554us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.441584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.443789 (+ 2205us) server_negotiation.cc:407] Connection header received
0806 01:11:18.443935 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.443942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.444163 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.444363 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:11:18.445026 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.446618 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.449121 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.450626 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.450752 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.450983 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.451522 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.451677 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17313,"thread_start_us":17145,"threads_started":1}
I20260806 01:11:18.452731 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.424805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.22.63.65:44887)
0806 01:11:18.440988 (+ 16183us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.440999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.443066 (+ 2067us) server_negotiation.cc:407] Connection header received
0806 01:11:18.443515 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.443523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.443721 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.443931 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:11:18.444862 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.446523 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.448785 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.450380 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.450531 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.450758 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.452236 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.452417 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15990,"thread_start_us":15853,"threads_started":1}
I20260806 01:11:18.451990 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.423545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46545 (local address 127.0.0.1:60964)
0806 01:11:18.443611 (+ 20066us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.443640 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.443794 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.444380 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.444386 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.444412 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:11:18.444823 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.444833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.446731 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.446736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.448951 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.448965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.451132 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.451138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.451319 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.451347 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.451430 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.451552 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19931,"mutex_wait_us":4,"thread_start_us":18937,"threads_started":1}
I20260806 01:11:18.455428 15359 tablet_service.cc:1467] Processing CreateTablet for tablet e96697c660a542d5afc451c42d0a92d3 (DEFAULT_TABLE table=test-table [id=678104784c304294a2997f5f134dda05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:11:18.456262 15159 tablet_service.cc:1467] Processing CreateTablet for tablet e96697c660a542d5afc451c42d0a92d3 (DEFAULT_TABLE table=test-table [id=678104784c304294a2997f5f134dda05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:11:18.457044 15359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96697c660a542d5afc451c42d0a92d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:11:18.457397 15159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96697c660a542d5afc451c42d0a92d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:11:18.457413 15553 tablet_service.cc:1467] Processing CreateTablet for tablet e96697c660a542d5afc451c42d0a92d3 (DEFAULT_TABLE table=test-table [id=678104784c304294a2997f5f134dda05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:11:18.458714 15553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e96697c660a542d5afc451c42d0a92d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:11:18.468720 15677 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Bootstrap starting.
I20260806 01:11:18.472013 15677 tablet_bootstrap.cc:654] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Neither blocks nor log segments found. Creating new log.
I20260806 01:11:18.473426 15677 log.cc:824] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:18.475958 15678 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Bootstrap starting.
I20260806 01:11:18.476558 15677 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: No bootstrap required, opened a new log
I20260806 01:11:18.476822 15677 ts_tablet_manager.cc:1397] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260806 01:11:18.479107 15678 tablet_bootstrap.cc:654] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Neither blocks nor log segments found. Creating new log.
I20260806 01:11:18.479326 15680 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Bootstrap starting.
I20260806 01:11:18.480249 15678 log.cc:824] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:18.482784 15680 tablet_bootstrap.cc:654] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Neither blocks nor log segments found. Creating new log.
I20260806 01:11:18.482895 15678 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: No bootstrap required, opened a new log
I20260806 01:11:18.483184 15678 ts_tablet_manager.cc:1397] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260806 01:11:18.484019 15680 log.cc:824] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:18.486285 15677 raft_consensus.cc:348] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.486756 15677 raft_consensus.cc:374] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:11:18.486958 15677 raft_consensus.cc:729] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aac8e51cd1045819df022a3200fb6d7, State: Initialized, Role: FOLLOWER
I20260806 01:11:18.487221 15680 tablet_bootstrap.cc:492] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: No bootstrap required, opened a new log
I20260806 01:11:18.487514 15680 ts_tablet_manager.cc:1397] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260806 01:11:18.487627 15677 consensus_queue.cc:260] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.489455 15677 ts_tablet_manager.cc:1428] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 01:11:18.493046 15678 raft_consensus.cc:348] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.493511 15678 raft_consensus.cc:374] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:11:18.493631 15678 raft_consensus.cc:729] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417d970652e54ef28d1a217ae7b79c3b, State: Initialized, Role: FOLLOWER
I20260806 01:11:18.494277 15678 consensus_queue.cc:260] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.496068 15680 raft_consensus.cc:348] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.496411 15680 raft_consensus.cc:374] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:11:18.496588 15680 raft_consensus.cc:729] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98b8c87134664785990de983ef880cb5, State: Initialized, Role: FOLLOWER
I20260806 01:11:18.497113 15444 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:11:18.497375 15680 consensus_queue.cc:260] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.497664 15678 ts_tablet_manager.cc:1428] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260806 01:11:18.498528 15653 heartbeater.cc:503] Master 127.22.63.126:37985 was elected leader, sending a full tablet report...
I20260806 01:11:18.498992 15680 ts_tablet_manager.cc:1428] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
W20260806 01:11:18.591764 15655 tablet.cc:2369] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:11:18.617081 15445 tablet.cc:2369] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:11:18.619091 15690 raft_consensus.cc:482] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:11:18.619323 15690 raft_consensus.cc:504] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.621021 15690 leader_election.cc:288] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887), 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545)
I20260806 01:11:18.627768 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.621325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:45069 (local address 127.22.63.65:44887)
0806 01:11:18.621582 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.621592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.622326 (+ 734us) server_negotiation.cc:407] Connection header received
0806 01:11:18.622544 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.622549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.622613 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.622689 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:11:18.623254 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.624180 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.625975 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.626806 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.626985 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.627065 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.627345 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.627493 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:11:18.627856 15694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.621250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.22.63.66:45069)
0806 01:11:18.622184 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.622208 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.622390 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.622712 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.622718 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.622745 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:11:18.623087 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.623098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.624323 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.624332 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.625814 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.625827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.627028 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.627034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.627185 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.627210 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.627320 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.627451 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":928,"threads_started":1}
I20260806 01:11:18.628300 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.621751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:52585 (local address 127.22.63.67:46545)
0806 01:11:18.622009 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.622016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.622617 (+ 601us) server_negotiation.cc:407] Connection header received
0806 01:11:18.622833 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.622839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.622911 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.622981 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:11:18.623711 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.624775 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.626428 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.627221 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.627393 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.627470 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.627985 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.628105 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:11:18.628844 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.621345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:46545 (local address 127.22.63.66:52585)
0806 01:11:18.622502 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.622519 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:18.622665 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:18.623068 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:18.623074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:18.623097 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:11:18.623533 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.623546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.624900 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.624907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.626271 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:18.626283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.627526 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.627534 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:18.627774 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.627812 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 01:11:18.628509 (+ 697us) client_negotiation.cc:240] Negotiation successful
0806 01:11:18.628633 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"thread_start_us":450,"threads_started":1}
I20260806 01:11:18.628830 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96697c660a542d5afc451c42d0a92d3" candidate_uuid: "417d970652e54ef28d1a217ae7b79c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aac8e51cd1045819df022a3200fb6d7" is_pre_election: true
I20260806 01:11:18.629375 15188 raft_consensus.cc:2454] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 417d970652e54ef28d1a217ae7b79c3b in term 0.
I20260806 01:11:18.629822 15583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96697c660a542d5afc451c42d0a92d3" candidate_uuid: "417d970652e54ef28d1a217ae7b79c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b8c87134664785990de983ef880cb5" is_pre_election: true
I20260806 01:11:18.630228 15291 leader_election.cc:302] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417d970652e54ef28d1a217ae7b79c3b, 7aac8e51cd1045819df022a3200fb6d7; no voters:
I20260806 01:11:18.630390 15583 raft_consensus.cc:2454] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 417d970652e54ef28d1a217ae7b79c3b in term 0.
I20260806 01:11:18.630715 15690 raft_consensus.cc:2781] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:11:18.630890 15690 raft_consensus.cc:482] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:11:18.631073 15690 raft_consensus.cc:3037] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:11:18.633312 15690 raft_consensus.cc:504] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.633932 15690 leader_election.cc:288] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [CANDIDATE]: Term 1 election: Requested vote from peers 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887), 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545)
I20260806 01:11:18.634493 15583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96697c660a542d5afc451c42d0a92d3" candidate_uuid: "417d970652e54ef28d1a217ae7b79c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98b8c87134664785990de983ef880cb5"
I20260806 01:11:18.634496 15188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e96697c660a542d5afc451c42d0a92d3" candidate_uuid: "417d970652e54ef28d1a217ae7b79c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aac8e51cd1045819df022a3200fb6d7"
I20260806 01:11:18.634714 15188 raft_consensus.cc:3037] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:11:18.634714 15583 raft_consensus.cc:3037] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 0 FOLLOWER]: Advancing to term 1
W20260806 01:11:18.637677 15264 tablet.cc:2369] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:11:18.638053 15188 raft_consensus.cc:2454] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 417d970652e54ef28d1a217ae7b79c3b in term 1.
I20260806 01:11:18.638054 15583 raft_consensus.cc:2454] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 417d970652e54ef28d1a217ae7b79c3b in term 1.
I20260806 01:11:18.638511 15293 leader_election.cc:302] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 417d970652e54ef28d1a217ae7b79c3b, 98b8c87134664785990de983ef880cb5; no voters:
I20260806 01:11:18.638978 15690 raft_consensus.cc:2781] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:11:18.639385 15690 raft_consensus.cc:686] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 1 LEADER]: Becoming Leader. State: Replica: 417d970652e54ef28d1a217ae7b79c3b, State: Running, Role: LEADER
I20260806 01:11:18.639979 15690 consensus_queue.cc:237] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } }
I20260806 01:11:18.644784 14983 catalog_manager.cc:5495] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66). New cstate: current_term: 1 leader_uuid: "417d970652e54ef28d1a217ae7b79c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "417d970652e54ef28d1a217ae7b79c3b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 39585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 } health_report { overall_health: UNKNOWN } } }
I20260806 01:11:18.682943 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.677627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39206 (local address 127.22.63.66:39585)
0806 01:11:18.677841 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.677845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.677990 (+ 145us) server_negotiation.cc:407] Connection header received
0806 01:11:18.678194 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.678198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.678275 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.678399 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:11:18.678743 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.679445 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.680858 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.681242 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.681335 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.681818 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:11:18.682295 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:11:18.682413 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.682558 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.682624 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:11:18.691397 15583 raft_consensus.cc:1261] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Refusing update from remote peer 417d970652e54ef28d1a217ae7b79c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:11:18.691395 15188 raft_consensus.cc:1261] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Refusing update from remote peer 417d970652e54ef28d1a217ae7b79c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:11:18.692468 15696 consensus_queue.cc:1035] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "98b8c87134664785990de983ef880cb5" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 46545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:11:18.692914 15690 consensus_queue.cc:1035] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aac8e51cd1045819df022a3200fb6d7" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:11:18.708007 15703 mvcc.cc:205] Tried to move back new op lower bound from 7315368667909722112 to 7315368667713249280. Current Snapshot: MvccSnapshot[applied={T|T < 7315368667909722112}]
I20260806 01:11:18.718233 14983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:11:18.718545 14983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:11:18.722311 14983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Sending DeleteTablet for 3 replicas of tablet e96697c660a542d5afc451c42d0a92d3
I20260806 01:11:18.723495 15159 tablet_service.cc:1514] Processing DeleteTablet for tablet e96697c660a542d5afc451c42d0a92d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:11:18 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:35922
I20260806 01:11:18.723634 15359 tablet_service.cc:1514] Processing DeleteTablet for tablet e96697c660a542d5afc451c42d0a92d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:11:18 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:39196
I20260806 01:11:18.723978 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 15034
I20260806 01:11:18.724332 15553 tablet_service.cc:1514] Processing DeleteTablet for tablet e96697c660a542d5afc451c42d0a92d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:11:18 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:60964
I20260806 01:11:18.733144 15712 tablet_replica.cc:330] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: stopping tablet replica
I20260806 01:11:18.733337 15711 tablet_replica.cc:330] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: stopping tablet replica
I20260806 01:11:18.733749 15712 raft_consensus.cc:2229] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:11:18.733769 15711 raft_consensus.cc:2229] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:11:18.734297 15711 raft_consensus.cc:2258] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:11:18.734917 15712 raft_consensus.cc:2258] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:11:18.736069 15717 tablet_replica.cc:330] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: stopping tablet replica
I20260806 01:11:18.736505 15717 raft_consensus.cc:2229] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:11:18.736994 15717 raft_consensus.cc:2258] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:11:18.737000 15711 ts_tablet_manager.cc:1905] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:11:18.739432 15717 ts_tablet_manager.cc:1905] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:11:18.741200 15712 ts_tablet_manager.cc:1905] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:11:18.743935 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.725044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35934 (local address 127.22.63.65:44887)
0806 01:11:18.725221 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.725228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.725246 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:11:18.725301 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.725306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.725381 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.725475 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:18.727631 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.728909 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.731040 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.731681 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.731844 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.735812 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:18.735828 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:18.735846 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:18.735891 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:18.739395 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.740372 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.740384 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.740389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.740558 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.741040 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.741045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.741048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.741654 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:18.741991 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.743208 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.743464 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:11:18.744891 15257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:18.749380 15712 ts_tablet_manager.cc:1918] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:11:18.749594 15712 log.cc:1196] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/e96697c660a542d5afc451c42d0a92d3
I20260806 01:11:18.750124 15712 ts_tablet_manager.cc:1939] T e96697c660a542d5afc451c42d0a92d3 P 417d970652e54ef28d1a217ae7b79c3b: Deleting consensus metadata
I20260806 01:11:18.751775 14963 catalog_manager.cc:4841] TS 417d970652e54ef28d1a217ae7b79c3b (127.22.63.66:39585): tablet e96697c660a542d5afc451c42d0a92d3 (table test-table [id=678104784c304294a2997f5f134dda05]) successfully deleted
I20260806 01:11:18.752540 15717 ts_tablet_manager.cc:1918] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:11:18.752744 15717 log.cc:1196] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/e96697c660a542d5afc451c42d0a92d3
I20260806 01:11:18.753242 15717 ts_tablet_manager.cc:1939] T e96697c660a542d5afc451c42d0a92d3 P 98b8c87134664785990de983ef880cb5: Deleting consensus metadata
I20260806 01:11:18.754747 14964 catalog_manager.cc:4841] TS 98b8c87134664785990de983ef880cb5 (127.22.63.67:46545): tablet e96697c660a542d5afc451c42d0a92d3 (table test-table [id=678104784c304294a2997f5f134dda05]) successfully deleted
I20260806 01:11:18.857041 15711 ts_tablet_manager.cc:1918] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:11:18.857313 15711 log.cc:1196] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/e96697c660a542d5afc451c42d0a92d3
I20260806 01:11:18.857909 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 15034
I20260806 01:11:18.857935 15711 ts_tablet_manager.cc:1939] T e96697c660a542d5afc451c42d0a92d3 P 7aac8e51cd1045819df022a3200fb6d7: Deleting consensus metadata
I20260806 01:11:18.890151 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 15269
W20260806 01:11:18.890139 14962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:44887 (error 108)
I20260806 01:11:18.892709 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.892139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35938)
0806 01:11:18.892394 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.892526 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260806 01:11:18.892997 14962 catalog_manager.cc:4584] TS 7aac8e51cd1045819df022a3200fb6d7 (127.22.63.65:44887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e96697c660a542d5afc451c42d0a92d3: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
I20260806 01:11:18.907397 15670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.891679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39210 (local address 127.22.63.66:39585)
0806 01:11:18.891919 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:18.891926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:18.891947 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:11:18.892016 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:18.892022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:18.892108 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:18.892219 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:18.894153 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.895571 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.897359 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:18.898065 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:18.898209 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:18.901089 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:18.901101 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:18.901115 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:18.901168 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:18.904304 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.905027 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.905035 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.905037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.905187 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:18.905603 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:18.905608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:18.905611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:18.906044 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:18.906399 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:18.906701 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 01:11:18.906965 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 01:11:18.908099 15439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:18.946942 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.946347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35946)
0806 01:11:18.946605 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.946720 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 01:11:18.948194 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:18.947731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35954)
0806 01:11:18.947944 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:18.948036 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:11:19.008826 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.008242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35970)
0806 01:11:19.008503 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:19.008636 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 01:11:19.009986 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.009576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35972)
0806 01:11:19.009756 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:19.009845 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 01:11:19.021687 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 15269
I20260806 01:11:19.046166 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 15447
I20260806 01:11:19.060655 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.047024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.22.63.67:46545)
0806 01:11:19.047237 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:19.047244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:19.047309 (+ 65us) server_negotiation.cc:407] Connection header received
0806 01:11:19.047521 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:19.047533 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:19.047613 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:19.047726 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:19.049259 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:19.049994 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:19.051309 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:19.051700 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:19.051881 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:19.054426 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:19.054441 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:19.054459 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:19.054494 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:19.057325 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:19.058224 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:19.058234 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:19.058238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:19.058430 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:19.058952 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:19.058960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:19.058963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:19.059404 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:19.059756 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:19.060070 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 01:11:19.060287 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:11:19.061432 15645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:19.090854 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.090259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35980)
0806 01:11:19.090516 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:19.090633 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 01:11:19.092046 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.091583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35984)
0806 01:11:19.091824 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:19.091907 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 01:11:19.151304 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 15447
I20260806 01:11:19.176735 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 14940
I20260806 01:11:19.178102 15026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:19.292745 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 14940
I20260806 01:11:19.292848 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:19.292012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:44887 (local address 127.0.0.1:35988)
0806 01:11:19.292457 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:19.292580 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:44887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260806 01:11:39.321677 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--webserver_interface=127.22.63.126
--webserver_port=37389
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:11:39.674432 15729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:39.674742 15729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:39.674834 15729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:39.682055 15729 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 01:11:39.682163 15729 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 01:11:39.682199 15729 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 01:11:39.682245 15729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:11:39.682305 15729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:39.682376 15729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:39.682420 15729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:11:39.682472 15729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:11:39.692392 15729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=37389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:39.693800 15729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:39.695112 15729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:39.707615 15741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:39.707592 15736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:39.707701 15738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:39.707727 15729 server_base.cc:1034] running on GCE node
I20260806 01:11:39.708968 15729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:39.710322 15729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:39.711582 15729 hybrid_clock.cc:648] HybridClock initialized: now 1785978699711524 us; error 34 us; skew 500 ppm
I20260806 01:11:39.711966 15729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:39 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978699, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:39.719695 15729 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:11:39.721239 15729 webserver.cc:460] Webserver started at http://127.22.63.126:37389/ using document root <none> and password file <none>
I20260806 01:11:39.721979 15729 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:39.722119 15729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:39.727806 15729 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:11:39.730854 15750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:39.732287 15729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 01:11:39.732491 15729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a1e450e271ff441e9d33886d4e8e7d1e"
format_stamp: "Formatted at 2026-08-06 01:10:53 on dist-test-slave-7f01"
server_key: "fc05d563ed2878c20d4521f16aba1e64"
server_key_iv: "544811c107e7f612d7bac3bb7f5341ec"
server_key_version: "encryptionkey@0"
I20260806 01:11:39.733466 15729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:39.762245 15729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:39.766518 15729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:39.766841 15729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:39.786156 15729 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37985
I20260806 01:11:39.786194 15817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37985 every 8 connection(s)
I20260806 01:11:39.787690 15729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:11:39.794613 15818 sys_catalog.cc:263] Verifying existing consensus state
I20260806 01:11:39.795588 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 15729
I20260806 01:11:39.796669 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:44887
--local_ip_for_outbound_sockets=127.22.63.65
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=37917
--webserver_interface=127.22.63.65
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:11:39.799571 15818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap starting.
I20260806 01:11:39.823292 15818 log.cc:824] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:39.840044 15818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 01:11:39.840620 15818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Bootstrap complete.
I20260806 01:11:39.850113 15818 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:39.850981 15818 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Initialized, Role: FOLLOWER
I20260806 01:11:39.851620 15818 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:39.851922 15818 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:11:39.852061 15818 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:11:39.852241 15818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 2 FOLLOWER]: Advancing to term 3
I20260806 01:11:39.856073 15818 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:39.856578 15818 leader_election.cc:302] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1e450e271ff441e9d33886d4e8e7d1e; no voters:
I20260806 01:11:39.857276 15818 leader_election.cc:288] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 01:11:39.857438 15825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 3 FOLLOWER]: Leader election won for term 3
I20260806 01:11:39.858804 15825 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [term 3 LEADER]: Becoming Leader. State: Replica: a1e450e271ff441e9d33886d4e8e7d1e, State: Running, Role: LEADER
I20260806 01:11:39.859367 15825 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } }
I20260806 01:11:39.859692 15818 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:11:39.862131 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1e450e271ff441e9d33886d4e8e7d1e. Latest consensus state: current_term: 3 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:11:39.862512 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:39.863204 15827 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1e450e271ff441e9d33886d4e8e7d1e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37985 } } }
I20260806 01:11:39.863534 15827 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:39.867009 15834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:11:39.883755 15834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:11:39.885396 15834 catalog_manager.cc:1259] Loaded cluster ID: f6b7e28941e44b928e3bcf6cf7ef14ea
I20260806 01:11:39.885521 15834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:11:39.889948 15834 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:11:39.891698 15834 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Loaded TSK: 0
I20260806 01:11:39.891906 15834 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1e450e271ff441e9d33886d4e8e7d1e: Loaded TSK: 1
F20260806 01:11:39.893045 15834 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f96fdda89ad google::LogMessage::Fail() at ??:0
@ 0x7f96fddab88a google::LogMessage::SendToLog() at ??:0
I20260806 01:11:39.899048 15852 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f96fdda84de google::LogMessage::Flush() at ??:0
@ 0x7f96fdda9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9701b0c7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9701afb73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f97137d71c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f97137d6783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f97138622a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f97138620c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f97138ac5e9 std::function<>::operator()() at ??:0
@ 0x7f97137dacdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f97137d8c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9713860610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9713860439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97009f4345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97009f791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f97009f7799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97009d2eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9703cac6db start_thread at ??:0
@ 0x7f96fcc4261f clone at ??:0
W20260806 01:11:40.201442 15821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:40.201777 15821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:40.201967 15821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:40.209486 15821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:40.209656 15821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:40.209838 15821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:11:40.219833 15821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:44887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=37917
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:40.221253 15821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:40.222743 15821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:40.236388 15895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:40.236871 15896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:40.236912 15898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:40.238076 15821 server_base.cc:1034] running on GCE node
I20260806 01:11:40.238673 15821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:40.239616 15821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:40.240885 15821 hybrid_clock.cc:648] HybridClock initialized: now 1785978700240842 us; error 71 us; skew 500 ppm
I20260806 01:11:40.241216 15821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:40 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:40.244702 15821 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:11:40.246250 15821 webserver.cc:460] Webserver started at http://127.22.63.65:37917/ using document root <none> and password file <none>
I20260806 01:11:40.246992 15821 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:40.247145 15821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:40.252767 15821 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 01:11:40.255519 15908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.256855 15821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:11:40.257074 15821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "7aac8e51cd1045819df022a3200fb6d7"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "421a94554c95b45cda83b422a3fd5581"
server_key_iv: "88fc40356b5e729ef5b182273e899954"
server_key_version: "encryptionkey@0"
I20260806 01:11:40.258081 15821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:40.273183 15821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:40.277586 15821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:40.277915 15821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:40.279011 15821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:40.280499 15821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:40.280635 15821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.280773 15821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:40.280872 15821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.317040 15821 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:44887
I20260806 01:11:40.317134 16072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:44887 every 8 connection(s)
I20260806 01:11:40.318609 15821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:11:40.323431 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.321581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:51709)
0806 01:11:40.322325 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.322558 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":298,"threads_started":1}
I20260806 01:11:40.324086 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 15821
W20260806 01:11:40.324306 16074 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:37985 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:37985: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
I20260806 01:11:40.324882 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:39585
--local_ip_for_outbound_sockets=127.22.63.66
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=33759
--webserver_interface=127.22.63.66
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:11:40.325251 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.324832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:51339)
0806 01:11:40.325024 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.325125 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:11:40.326118 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.325773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:51921)
0806 01:11:40.325930 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.326014 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:11:40.326393 16074 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:11:40.671151 16078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:40.671432 16078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:40.671527 16078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:40.678804 16078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:40.678925 16078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:40.679039 16078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:11:40.688839 16078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:39585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=33759
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:40.690056 16078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:40.691336 16078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:40.704543 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:40.704617 16085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:40.704643 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:40.705571 16078 server_base.cc:1034] running on GCE node
I20260806 01:11:40.706358 16078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:40.707435 16078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:40.708703 16078 hybrid_clock.cc:648] HybridClock initialized: now 1785978700708636 us; error 69 us; skew 500 ppm
I20260806 01:11:40.709082 16078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:40 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978700, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:40.712625 16078 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:11:40.714375 16078 webserver.cc:460] Webserver started at http://127.22.63.66:33759/ using document root <none> and password file <none>
I20260806 01:11:40.715140 16078 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:40.715307 16078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:40.721287 16078 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 01:11:40.724206 16099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.725592 16078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:11:40.725773 16078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "417d970652e54ef28d1a217ae7b79c3b"
format_stamp: "Formatted at 2026-08-06 01:10:54 on dist-test-slave-7f01"
server_key: "93f61bf5d44d9bd4e00bbe48378e2178"
server_key_iv: "c8360b62e227c315cb3fbaf287bdbcca"
server_key_version: "encryptionkey@0"
I20260806 01:11:40.726871 16078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:40.742671 16078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:40.747036 16078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:40.747326 16078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:40.748435 16078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:40.749840 16078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:40.749984 16078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.750106 16078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:40.750200 16078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:40.785951 16078 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:39585
I20260806 01:11:40.786047 16259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:39585 every 8 connection(s)
I20260806 01:11:40.787503 16078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:11:40.788113 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 16078
I20260806 01:11:40.789062 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:46545
--local_ip_for_outbound_sockets=127.22.63.67
--tserver_master_addrs=127.22.63.126:37985
--webserver_port=35147
--webserver_interface=127.22.63.67
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:33287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:11:40.792763 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.790695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:40063)
0806 01:11:40.791559 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.791860 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":365,"threads_started":1}
W20260806 01:11:40.793557 16261 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:37985 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:37985: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
I20260806 01:11:40.794406 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.794073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:48137)
0806 01:11:40.794226 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.794306 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 01:11:40.795358 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:40.795046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:42037)
0806 01:11:40.795175 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:40.795248 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:11:40.795656 16261 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 01:11:41.133522 16264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:41.133816 16264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:41.133912 16264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:41.141093 16264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:41.141209 16264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:41.141323 16264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:11:41.152848 16264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:33287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:46545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=35147
--tserver_master_addrs=127.22.63.126:37985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:41.154222 16264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:41.155570 16264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:41.169334 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:41.169440 16274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:41.169365 16276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:41.170596 16264 server_base.cc:1034] running on GCE node
I20260806 01:11:41.171213 16264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:41.172230 16264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:41.173466 16264 hybrid_clock.cc:648] HybridClock initialized: now 1785978701173433 us; error 54 us; skew 500 ppm
I20260806 01:11:41.173784 16264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:41 dist-test-slave-7f01 krb5kdc[14070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978701, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:41.177119 16264 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:11:41.178494 16264 webserver.cc:460] Webserver started at http://127.22.63.67:35147/ using document root <none> and password file <none>
I20260806 01:11:41.179141 16264 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:41.179243 16264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:41.184625 16264 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:11:41.187266 16287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:41.188665 16264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:11:41.188805 16264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "98b8c87134664785990de983ef880cb5"
format_stamp: "Formatted at 2026-08-06 01:10:55 on dist-test-slave-7f01"
server_key: "bdd022764d1f32d912af07b204e30044"
server_key_iv: "9a3941b1de2f5258bcc5ca6af2cd6cce"
server_key_version: "encryptionkey@0"
I20260806 01:11:41.189741 16264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:41.222826 16264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:41.227319 16264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:41.227560 16264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:41.228560 16264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:41.229998 16264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:41.230084 16264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:41.230173 16264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:41.230228 16264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:41.264565 16264 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:46545
I20260806 01:11:41.264664 16451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:46545 every 8 connection(s)
I20260806 01:11:41.266091 16264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:11:41.270857 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:41.269021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:40735)
0806 01:11:41.269755 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:41.270000 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":284,"threads_started":1}
W20260806 01:11:41.271610 16452 heartbeater.cc:644] Failed to heartbeat to 127.22.63.126:37985 (0 consecutive failures): Network error: Failed to ping master at 127.22.63.126:37985: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
I20260806 01:11:41.271999 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 16264
I20260806 01:11:41.272745 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:41.272351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:52601)
0806 01:11:41.272538 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:41.272638 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:11:41.273768 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:41.273384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:35691)
0806 01:11:41.273544 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:41.273623 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 01:11:41.274066 16452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 01:11:41.327916 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:41.326998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:55169)
0806 01:11:41.327498 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:41.327614 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":234,"threads_started":1}
I20260806 01:11:41.796959 16458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:41.796188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:37195)
0806 01:11:41.796609 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:41.796714 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":206,"threads_started":1}
I20260806 01:11:42.275382 16459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:42.274587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:49409)
0806 01:11:42.275045 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:42.275138 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260806 01:11:42.329450 16460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:42.328769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:38245)
0806 01:11:42.329202 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:42.329281 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":223,"threads_started":1}
I20260806 01:11:42.798650 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:42.797887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:49541)
0806 01:11:42.798377 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:42.798464 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260806 01:11:43.277159 16463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:43.276337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:49433)
0806 01:11:43.276886 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:43.276979 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":271,"threads_started":1}
I20260806 01:11:43.331168 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:43.330381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:33221)
0806 01:11:43.330893 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:43.330976 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
I20260806 01:11:43.800462 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:43.799623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:44181)
0806 01:11:43.800181 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:43.800267 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":316,"threads_started":1}
I20260806 01:11:44.278975 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:44.278183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:58531)
0806 01:11:44.278676 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:44.278761 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":267,"threads_started":1}
I20260806 01:11:44.332893 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:44.332108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:57449)
0806 01:11:44.332572 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:44.332656 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":220,"threads_started":1}
I20260806 01:11:44.802179 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:44.801429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:53339)
0806 01:11:44.801887 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:44.801973 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":232,"threads_started":1}
I20260806 01:11:45.280736 16475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:45.279999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:54841)
0806 01:11:45.280480 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:45.280562 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":256,"threads_started":1}
I20260806 01:11:45.334715 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:45.333874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:33993)
0806 01:11:45.334419 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:45.334524 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":277,"threads_started":1}
I20260806 01:11:45.804020 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:45.803113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:43731)
0806 01:11:45.803642 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:45.803816 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":275,"threads_started":1}
I20260806 01:11:46.282687 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:46.281836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:35233)
0806 01:11:46.282421 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:46.282507 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":337,"threads_started":1}
I20260806 01:11:46.336514 16483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:46.335673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:35925)
0806 01:11:46.336222 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:46.336330 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":307,"threads_started":1}
I20260806 01:11:46.805961 16484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:46.805148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:39089)
0806 01:11:46.805659 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:46.805766 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":261,"threads_started":1}
I20260806 01:11:47.284538 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:47.283683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:51325)
0806 01:11:47.284258 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:47.284363 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":340,"threads_started":1}
I20260806 01:11:47.338367 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:47.337503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:41781)
0806 01:11:47.338037 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:47.338123 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":260,"threads_started":1}
I20260806 01:11:47.808037 16489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:47.806985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:47563)
0806 01:11:47.807527 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:47.807612 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":254,"threads_started":1}
I20260806 01:11:48.286708 16490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:48.285578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:50939)
0806 01:11:48.286285 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:48.286407 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":285,"threads_started":1}
I20260806 01:11:48.340116 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:48.339346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:48811)
0806 01:11:48.339842 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:48.339929 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260806 01:11:48.809792 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:48.809076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:50317)
0806 01:11:48.809529 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:48.809617 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":235,"threads_started":1}
I20260806 01:11:49.288646 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:49.287857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:46615)
0806 01:11:49.288388 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:49.288475 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":300,"threads_started":1}
I20260806 01:11:49.341935 16497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:49.341109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:35209)
0806 01:11:49.341644 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:49.341729 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":277,"threads_started":1}
I20260806 01:11:49.811902 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:49.810924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:53103)
0806 01:11:49.811520 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:49.811619 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":282,"threads_started":1}
I20260806 01:11:50.290438 16500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:50.289621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:36267)
0806 01:11:50.290145 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:50.290260 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":265,"threads_started":1}
I20260806 01:11:50.343611 16501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:50.342885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:49579)
0806 01:11:50.343352 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:50.343431 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":279,"threads_started":1}
I20260806 01:11:50.813637 16502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:50.812893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:34735)
0806 01:11:50.813371 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:50.813457 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260806 01:11:51.292292 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:51.291486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:39479)
0806 01:11:51.292032 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:51.292123 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":331,"threads_started":1}
I20260806 01:11:51.345314 16504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:51.344555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:53153)
0806 01:11:51.345031 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:51.345122 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":265,"threads_started":1}
I20260806 01:11:51.815339 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:51.814570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:33139)
0806 01:11:51.815059 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:51.815154 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":264,"threads_started":1}
I20260806 01:11:52.294088 16508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:52.293210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:54647)
0806 01:11:52.293783 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:52.293900 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":337,"threads_started":1}
I20260806 01:11:52.347060 16509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:52.346238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:48173)
0806 01:11:52.346794 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:52.346877 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":323,"threads_started":1}
I20260806 01:11:52.817008 16510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:52.816283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:57779)
0806 01:11:52.816740 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:52.816822 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260806 01:11:53.295936 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:53.295083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:45961)
0806 01:11:53.295588 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:53.295675 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":272,"threads_started":1}
I20260806 01:11:53.348841 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:53.348061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:36471)
0806 01:11:53.348549 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:53.348643 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":245,"threads_started":1}
I20260806 01:11:53.818719 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:53.817917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:42141)
0806 01:11:53.818419 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:53.818519 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":251,"threads_started":1}
I20260806 01:11:54.297673 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:54.296874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:50925)
0806 01:11:54.297384 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:54.297480 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":263,"threads_started":1}
I20260806 01:11:54.350607 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:54.349861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:56659)
0806 01:11:54.350321 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:54.350418 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":256,"threads_started":1}
I20260806 01:11:54.820574 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:54.819698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:34617)
0806 01:11:54.820269 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:54.820363 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":284,"threads_started":1}
I20260806 01:11:55.299352 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:55.298584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:41465)
0806 01:11:55.299053 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:55.299155 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
I20260806 01:11:55.352531 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:55.351795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.65:44187)
0806 01:11:55.352242 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:55.352330 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":239,"threads_started":1}
I20260806 01:11:55.822428 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:55.821614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.66:50375)
0806 01:11:55.822135 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:55.822227 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":300,"threads_started":1}
I20260806 01:11:56.276474 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 15821
I20260806 01:11:56.294852 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:56.277560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.22.63.65:44887)
0806 01:11:56.278189 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:56.278210 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:56.278402 (+ 192us) server_negotiation.cc:407] Connection header received
0806 01:11:56.280422 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:56.280431 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:56.280871 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:56.281156 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:56.282535 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.283897 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.285201 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.285877 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.286009 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:56.288709 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:56.288717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:56.288731 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:56.288771 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:56.291728 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.292458 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.292464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.292471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.292632 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.292999 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.293007 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.293012 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.293533 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:56.293747 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:56.294183 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 01:11:56.294563 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":294,"threads_started":1}
I20260806 01:11:56.296278 16064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:56.301465 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:56.300589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37985 (local address 127.22.63.67:56609)
0806 01:11:56.301133 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:56.301245 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.126:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":257,"threads_started":1}
I20260806 01:11:56.380909 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 15821
I20260806 01:11:56.405079 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 16078
I20260806 01:11:56.424101 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:56.406042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.22.63.66:39585)
0806 01:11:56.406640 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:56.406653 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:56.406692 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:11:56.408464 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:56.408474 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:56.408902 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:56.409248 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:56.410715 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.412141 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.413497 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.414164 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.414391 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:56.417113 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:56.417121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:56.417136 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:56.417173 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:56.420341 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.421146 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.421159 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.421167 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.421356 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.421791 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.421797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.421801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.422368 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:56.422704 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:56.423227 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 01:11:56.423703 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
I20260806 01:11:56.425601 16254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:56.509599 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 16078
I20260806 01:11:56.535000 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 16264
I20260806 01:11:56.553655 16539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:56.536145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45206 (local address 127.22.63.67:46545)
0806 01:11:56.536723 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:56.536738 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:56.536782 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:11:56.538495 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:56.538506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:56.538954 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:56.539260 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:56.540810 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.542096 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.543441 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:56.544206 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:56.544357 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:56.547099 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:56.547107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:56.547125 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:56.547167 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:56.550167 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.550916 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.550923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.550929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.551104 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:56.551567 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:56.551571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:56.551574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:56.552258 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:56.552505 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:56.552884 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 01:11:56.553330 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":277,"threads_started":1}
I20260806 01:11:56.555263 16447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:11:56.639221 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 16264
2026-08-06T01:11:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65244 ms)
[----------] 30 tests from SecurityITest (313890 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:11:56 dist-test-slave-7f01 krb5kdc[16546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:11:56 dist-test-slave-7f01 krb5kdc[16546](info): set up 2 sockets
Aug 06 01:11:56 dist-test-slave-7f01 krb5kdc[16546](info): commencing operation
krb5kdc: starting...
W20260806 01:11:58.002763 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:11:58 dist-test-slave-7f01 krb5kdc[16546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:11:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:11:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:11:58.327253 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42243
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:11:58.683441 16562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:58.683847 16562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:58.684012 16562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:58.691512 16562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:11:58.691654 16562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:58.691787 16562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:58.691905 16562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:11:58.691982 16562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:11:58.701875 16562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:58.703552 16562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:58.705030 16562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:58.716889 16571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:58.716958 16572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:58.716933 16575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:58.718076 16562 server_base.cc:1034] running on GCE node
I20260806 01:11:58.718883 16562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:58.720412 16562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:58.721671 16562 hybrid_clock.cc:648] HybridClock initialized: now 1785978718721605 us; error 53 us; skew 500 ppm
I20260806 01:11:58.722051 16562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:58 dist-test-slave-7f01 krb5kdc[16546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:58.730743 16562 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:11:58.732533 16562 webserver.cc:460] Webserver started at http://127.22.63.126:36383/ using document root <none> and password file <none>
I20260806 01:11:58.733265 16562 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:58.733377 16562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:58.733659 16562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:11:58.737018 16562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "48785e8e78b64d88af56ad80839d0904"
format_stamp: "Formatted at 2026-08-06 01:11:58 on dist-test-slave-7f01"
server_key: "670e2a22342bf1d2dfaacedc84f13a4f"
server_key_iv: "26a8275eca8ad72b79b855e3868befbb"
server_key_version: "encryptionkey@0"
I20260806 01:11:58.737797 16562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "48785e8e78b64d88af56ad80839d0904"
format_stamp: "Formatted at 2026-08-06 01:11:58 on dist-test-slave-7f01"
server_key: "670e2a22342bf1d2dfaacedc84f13a4f"
server_key_iv: "26a8275eca8ad72b79b855e3868befbb"
server_key_version: "encryptionkey@0"
I20260806 01:11:58.743479 16562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 01:11:58.747366 16589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:58.748898 16562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:11:58.749114 16562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "48785e8e78b64d88af56ad80839d0904"
format_stamp: "Formatted at 2026-08-06 01:11:58 on dist-test-slave-7f01"
server_key: "670e2a22342bf1d2dfaacedc84f13a4f"
server_key_iv: "26a8275eca8ad72b79b855e3868befbb"
server_key_version: "encryptionkey@0"
I20260806 01:11:58.749339 16562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:58.778100 16562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:58.782805 16562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:58.783134 16562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:58.802147 16562 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:42243
I20260806 01:11:58.802239 16665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:42243 every 8 connection(s)
I20260806 01:11:58.803624 16562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:11:58.807852 16666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:11:58.813012 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 16562
I20260806 01:11:58.813284 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:11:58.813977 22781 external_mini_cluster.cc:1428] Setting key 4d2400081e01dbf8f580e4f6aedb1065
I20260806 01:11:58.821938 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: Bootstrap starting.
I20260806 01:11:58.826284 16666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: Neither blocks nor log segments found. Creating new log.
I20260806 01:11:58.827841 16666 log.cc:824] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: Log is configured to *not* fsync() on all Append() calls
I20260806 01:11:58.831053 16666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: No bootstrap required, opened a new log
Aug 06 01:11:58 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:11:58.842239 16669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:58.816202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.22.63.126:42243)
0806 01:11:58.817198 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:58.817215 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:58.817274 (+ 59us) server_negotiation.cc:407] Connection header received
0806 01:11:58.820159 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:58.820207 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:58.820790 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:58.821276 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:58.822782 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:58.825133 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:58.826596 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:58.827834 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:58.828034 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:58.832499 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:58.832511 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:58.832534 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:58.832586 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:58.836994 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:58.837821 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:58.837833 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:58.837843 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:58.838096 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:58.838487 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:58.838492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:58.838495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:58.839278 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:58.839574 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:58.840203 (+ 629us) server_negotiation.cc:299] Negotiation successful
0806 01:11:58.840740 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":415,"threads_started":1}
I20260806 01:11:58.842427 16666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } }
I20260806 01:11:58.842901 16666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:11:58.843050 16666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48785e8e78b64d88af56ad80839d0904, State: Initialized, Role: FOLLOWER
I20260806 01:11:58.843783 16666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } }
I20260806 01:11:58.844080 16666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:11:58.844244 16666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:11:58.844468 16666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:11:58.847056 16666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } }
I20260806 01:11:58.847718 16666 leader_election.cc:302] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48785e8e78b64d88af56ad80839d0904; no voters:
I20260806 01:11:58.848337 16666 leader_election.cc:288] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:11:58.848516 16671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:11:58.849217 16671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [term 1 LEADER]: Becoming Leader. State: Replica: 48785e8e78b64d88af56ad80839d0904, State: Running, Role: LEADER
I20260806 01:11:58.849795 16671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } }
I20260806 01:11:58.850508 16666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:11:58.852650 16675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48785e8e78b64d88af56ad80839d0904. Latest consensus state: current_term: 1 leader_uuid: "48785e8e78b64d88af56ad80839d0904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } } }
I20260806 01:11:58.852730 16674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48785e8e78b64d88af56ad80839d0904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48785e8e78b64d88af56ad80839d0904" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42243 } } }
I20260806 01:11:58.853260 16675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:58.853816 16674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904 [sys.catalog]: This master's current role is: LEADER
I20260806 01:11:58.860725 16684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:11:58.865779 16684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:11:58.874881 16684 catalog_manager.cc:1347] Generated new cluster ID: 7f6a2de35e804c4faaecc1cb64eecf26
I20260806 01:11:58.875012 16684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:11:58.885677 16684 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:11:58.887403 16684 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:11:58.898484 16684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: Generated new TSK 0
I20260806 01:11:58.899272 16684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:11:58.979611 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:11:59.332614 16710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:59.332947 16710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:59.333122 16710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:59.340225 16710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:59.340380 16710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:59.340572 16710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:11:59.350268 16710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:59.351824 16710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:59.353401 16710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:59.367444 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:59.367533 16715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:59.367509 16719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:59.368211 16710 server_base.cc:1034] running on GCE node
I20260806 01:11:59.368911 16710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:59.369911 16710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:59.371143 16710 hybrid_clock.cc:648] HybridClock initialized: now 1785978719371083 us; error 47 us; skew 500 ppm
I20260806 01:11:59.371470 16710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:59 dist-test-slave-7f01 krb5kdc[16546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978719, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:59.374627 16710 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:11:59.376210 16710 webserver.cc:460] Webserver started at http://127.22.63.65:44443/ using document root <none> and password file <none>
I20260806 01:11:59.376905 16710 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:59.377056 16710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:59.377390 16710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:11:59.380398 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "c46edfd6c30a41ea9ff48b67152fd400"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "64e883eda08a76cf1b25814d3efcfdfc"
server_key_iv: "9a2c00b15e8682174b092d1e8df756ea"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.381264 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "c46edfd6c30a41ea9ff48b67152fd400"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "64e883eda08a76cf1b25814d3efcfdfc"
server_key_iv: "9a2c00b15e8682174b092d1e8df756ea"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.386502 16710 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:11:59.390007 16735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:59.391346 16710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260806 01:11:59.391522 16710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "c46edfd6c30a41ea9ff48b67152fd400"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "64e883eda08a76cf1b25814d3efcfdfc"
server_key_iv: "9a2c00b15e8682174b092d1e8df756ea"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.391721 16710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:11:59.442507 16710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:11:59.446542 16710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:11:59.446861 16710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:11:59.447942 16710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:11:59.449323 16710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:11:59.449432 16710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:59.449585 16710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:11:59.449661 16710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:59.484220 16710 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:45603
I20260806 01:11:59.484278 16892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:45603 every 8 connection(s)
I20260806 01:11:59.485559 16710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:11:59.495568 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 16710
I20260806 01:11:59.495872 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:11:59.496686 22781 external_mini_cluster.cc:1428] Setting key 4ec2a9c78aa05ce5310fab6714d6d7d6
Aug 06 01:11:59 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978719, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:11:59.506366 16895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:59.488223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:46645 (local address 127.22.63.126:42243)
0806 01:11:59.488715 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 01:11:59.488720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:11:59.489558 (+ 838us) server_negotiation.cc:407] Connection header received
0806 01:11:59.491457 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:11:59.491463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:11:59.491565 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:11:59.491691 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:11:59.493207 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:59.493963 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:59.495959 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:11:59.496441 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:11:59.496598 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:59.500572 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:11:59.500577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:11:59.500580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:11:59.500612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:11:59.503179 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:59.504031 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:59.504036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:59.504038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:59.504156 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:11:59.504605 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:11:59.504610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:11:59.504612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:11:59.504867 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:11:59.505030 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 01:11:59.505752 (+ 722us) server_negotiation.cc:299] Negotiation successful
0806 01:11:59.505992 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":265,"threads_started":1}
I20260806 01:11:59.507306 16896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:11:59.488710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42243 (local address 127.22.63.65:46645)
0806 01:11:59.489390 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 01:11:59.489434 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:11:59.491176 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:11:59.491956 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:11:59.491963 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:11:59.492475 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:11:59.493029 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:59.493045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:59.494113 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:59.494118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:59.495730 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:11:59.495770 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:11:59.496724 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:11:59.496735 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:11:59.497041 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:11:59.497870 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:11:59.497901 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:11:59.500343 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:11:59.503325 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:59.503335 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:59.503352 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:59.503892 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:59.504271 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:11:59.504276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:11:59.504280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:11:59.504478 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:11:59.505000 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:11:59.505008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:11:59.505358 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 01:11:59.505630 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 01:11:59.506167 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":265,"threads_started":1}
I20260806 01:11:59.508936 16893 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42243
I20260806 01:11:59.509241 16893 heartbeater.cc:464] Registering TS with master...
I20260806 01:11:59.509954 16893 heartbeater.cc:511] Master 127.22.63.126:42243 requested a full tablet report, sending...
I20260806 01:11:59.512048 16612 ts_manager.cc:194] Registered new tserver with Master: c46edfd6c30a41ea9ff48b67152fd400 (127.22.63.65:45603)
I20260806 01:11:59.513514 16612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:46645
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:11:59.569206 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:11:59.926555 16902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:11:59.926867 16902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:11:59.926977 16902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:11:59.934316 16902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:11:59.934422 16902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:11:59.934533 16902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:11:59.944316 16902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:11:59.945577 16902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:11:59.946907 16902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:11:59.960842 16910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:59.960899 16911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:11:59.960803 16914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:11:59.961724 16902 server_base.cc:1034] running on GCE node
I20260806 01:11:59.962369 16902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:11:59.963312 16902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:11:59.964566 16902 hybrid_clock.cc:648] HybridClock initialized: now 1785978719964527 us; error 41 us; skew 500 ppm
I20260806 01:11:59.964936 16902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:11:59 dist-test-slave-7f01 krb5kdc[16546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978719, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:11:59.968242 16902 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:11:59.969729 16902 webserver.cc:460] Webserver started at http://127.22.63.66:37275/ using document root <none> and password file <none>
I20260806 01:11:59.970440 16902 fs_manager.cc:359] Metadata directory not provided
I20260806 01:11:59.970599 16902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:11:59.970927 16902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:11:59.973953 16902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "28a115c430cd4eac889c688527eb42f4"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "1e31558f8b8054e021cb8639652c83f9"
server_key_iv: "9a37f65b41474516cafe0faad15fc686"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.974746 16902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "28a115c430cd4eac889c688527eb42f4"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "1e31558f8b8054e021cb8639652c83f9"
server_key_iv: "9a37f65b41474516cafe0faad15fc686"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.980150 16902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260806 01:11:59.983632 16925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:11:59.984997 16902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:11:59.985172 16902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "28a115c430cd4eac889c688527eb42f4"
format_stamp: "Formatted at 2026-08-06 01:11:59 on dist-test-slave-7f01"
server_key: "1e31558f8b8054e021cb8639652c83f9"
server_key_iv: "9a37f65b41474516cafe0faad15fc686"
server_key_version: "encryptionkey@0"
I20260806 01:11:59.985400 16902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:00.020324 16902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:00.024490 16902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:00.024791 16902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:00.025799 16902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:00.027251 16902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:00.027359 16902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:00.027514 16902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:00.027586 16902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:00.062978 16902 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:38097
I20260806 01:12:00.063012 17083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:38097 every 8 connection(s)
I20260806 01:12:00.064443 16902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:00.073522 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 16902
I20260806 01:12:00.073736 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:00.074324 22781 external_mini_cluster.cc:1428] Setting key 341b7fa5a1aa7eca0be1ac134f06a9d3
Aug 06 01:12:00 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978719, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:00.088209 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.067201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:34895 (local address 127.22.63.126:42243)
0806 01:12:00.067676 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.067681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.068703 (+ 1022us) server_negotiation.cc:407] Connection header received
0806 01:12:00.070571 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.070576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.070658 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.070768 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:00.072349 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.073154 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.075472 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.076174 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.076302 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.080650 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:00.080659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:00.080667 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:00.080714 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:00.084408 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.085203 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.085212 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.085218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.085395 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.085890 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.085897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.085903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.086309 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:00.086466 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.087400 (+ 934us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.087678 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260806 01:12:00.089386 17088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.067823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42243 (local address 127.22.63.66:34895)
0806 01:12:00.068536 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.068581 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.070253 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.070973 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.070980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.071510 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:00.072165 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.072183 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.073330 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.073338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.075298 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.075313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.076444 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.076451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.076696 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.077468 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:00.077513 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:00.080394 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:00.084566 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:00.084574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:00.084588 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:00.085059 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:00.085533 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:00.085537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:00.085539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:00.085737 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:00.086437 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:00.086443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:00.086958 (+ 515us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.087256 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.087824 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":283,"threads_started":1}
I20260806 01:12:00.090974 17084 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42243
I20260806 01:12:00.091353 17084 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:00.092267 17084 heartbeater.cc:511] Master 127.22.63.126:42243 requested a full tablet report, sending...
I20260806 01:12:00.093796 16612 ts_manager.cc:194] Registered new tserver with Master: 28a115c430cd4eac889c688527eb42f4 (127.22.63.66:38097)
I20260806 01:12:00.094740 16612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:34895
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:12:00.148340 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:00.502171 17094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:00.502460 17094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:00.502556 17094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:00.509804 17094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:00.509955 17094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:00.510093 17094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:00.516390 16893 heartbeater.cc:503] Master 127.22.63.126:42243 was elected leader, sending a full tablet report...
I20260806 01:12:00.520749 17094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:00.522153 17094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:00.523543 17094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:00.537698 17110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:00.537869 17107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:00.537871 17106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:00.538954 17094 server_base.cc:1034] running on GCE node
I20260806 01:12:00.539714 17094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:00.540763 17094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:00.541986 17094 hybrid_clock.cc:648] HybridClock initialized: now 1785978720541912 us; error 61 us; skew 500 ppm
I20260806 01:12:00.542353 17094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:00 dist-test-slave-7f01 krb5kdc[16546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978720, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:00.545825 17094 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:12:00.547428 17094 webserver.cc:460] Webserver started at http://127.22.63.67:35833/ using document root <none> and password file <none>
I20260806 01:12:00.548277 17094 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:00.548441 17094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:00.548784 17094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:00.552170 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "07aa04f3a4894df89599883f69b32a0f"
format_stamp: "Formatted at 2026-08-06 01:12:00 on dist-test-slave-7f01"
server_key: "7dbf39c0883eb14517b760473c7f31b4"
server_key_iv: "e2e832871aafd98b3854c097cb6efa2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:00.553160 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "07aa04f3a4894df89599883f69b32a0f"
format_stamp: "Formatted at 2026-08-06 01:12:00 on dist-test-slave-7f01"
server_key: "7dbf39c0883eb14517b760473c7f31b4"
server_key_iv: "e2e832871aafd98b3854c097cb6efa2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:00.558940 17094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:12:00.562729 17120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:00.564338 17094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:00.564492 17094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "07aa04f3a4894df89599883f69b32a0f"
format_stamp: "Formatted at 2026-08-06 01:12:00 on dist-test-slave-7f01"
server_key: "7dbf39c0883eb14517b760473c7f31b4"
server_key_iv: "e2e832871aafd98b3854c097cb6efa2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:00.564682 17094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:00.601445 17094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:00.605664 17094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:00.605929 17094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:00.606900 17094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:00.608263 17094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:00.608356 17094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:00.608449 17094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:00.608496 17094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:00.643544 17094 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:35781
I20260806 01:12:00.643579 17283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:35781 every 8 connection(s)
I20260806 01:12:00.644956 17094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:00.652063 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 17094
I20260806 01:12:00.652279 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:00.652781 22781 external_mini_cluster.cc:1428] Setting key 579513eaa2149b6f3d9d4a6d16551b9e
Aug 06 01:12:00 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978720, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:00.669159 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.647948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:43557 (local address 127.22.63.126:42243)
0806 01:12:00.648730 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.648735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.649234 (+ 499us) server_negotiation.cc:407] Connection header received
0806 01:12:00.651258 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.651263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.651362 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.651525 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:00.653630 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.654616 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.656763 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.657170 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.657275 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.661860 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:00.661867 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:00.661871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:00.661911 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:00.664975 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.666025 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.666101 (+ 76us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.666109 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.666264 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.666939 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.666945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.666948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.667296 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:00.667480 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.668440 (+ 960us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.668687 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":448,"threads_started":1}
I20260806 01:12:00.670262 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.648219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:42243 (local address 127.22.63.67:43557)
0806 01:12:00.649059 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.649115 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.650978 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.651869 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.651887 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.652574 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:00.653393 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.653430 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.654809 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.654815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.656581 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.656595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.657413 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.657419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.657745 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.658509 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:00.658534 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:00.661439 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:00.665193 (+ 3754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:00.665202 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:00.665218 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:00.665771 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:00.666441 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:00.666446 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:00.666450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:00.666729 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:00.667446 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:00.667455 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:00.667966 (+ 511us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.668363 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.668988 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":362,"threads_started":1}
I20260806 01:12:00.671720 17285 heartbeater.cc:347] Connected to a master server at 127.22.63.126:42243
I20260806 01:12:00.672129 17285 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:00.672921 17285 heartbeater.cc:511] Master 127.22.63.126:42243 requested a full tablet report, sending...
I20260806 01:12:00.674402 16612 ts_manager.cc:194] Registered new tserver with Master: 07aa04f3a4894df89599883f69b32a0f (127.22.63.67:35781)
I20260806 01:12:00.675230 16612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:43557
I20260806 01:12:00.681485 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:00.700580 17287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.686300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.22.63.126:42243)
0806 01:12:00.686515 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.686521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.687044 (+ 523us) server_negotiation.cc:407] Connection header received
0806 01:12:00.687174 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.687178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.687250 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.687346 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:00.688988 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.689865 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.691271 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.691667 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.691827 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.694455 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:00.694461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:00.694464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:00.694502 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:00.697940 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.698674 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.698677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.698680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.698803 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.699235 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.699239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.699241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.699567 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:00.699835 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.700166 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.700345 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:12:00.704807 16612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:00.711959 16612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:00.737186 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.725669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.22.63.67:35781)
0806 01:12:00.726927 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.726936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.726975 (+ 39us) server_negotiation.cc:407] Connection header received
0806 01:12:00.727082 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.727089 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.727341 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.728419 (+ 1078us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:00.728759 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.730482 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.734452 (+ 3970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.735678 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.736292 (+ 614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.736573 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.736650 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.736819 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1060,"thread_start_us":896,"threads_started":1}
I20260806 01:12:00.737181 17301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.725281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:35781 (local address 127.0.0.1:51222)
0806 01:12:00.726020 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.726074 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.726244 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.727630 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.727641 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.727685 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:00.728159 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.728176 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.730674 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.730683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.733485 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.733506 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.735929 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.735937 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.736216 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.736375 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.736492 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.736692 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":56,"thread_start_us":308,"threads_started":1}
I20260806 01:12:00.741346 17187 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3b792ecc644efab8bef9519557f725 (DEFAULT_TABLE table=test-table [id=88c3ad1627604f56a59ed95b88674db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:00.742908 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.726572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:38097 (local address 127.0.0.1:57324)
0806 01:12:00.727191 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.727210 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.727321 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.735758 (+ 8437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.735765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.735800 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:00.736255 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.736268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.738119 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.738124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.740155 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.740173 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.742091 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.742098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.742283 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.742334 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.742402 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.742531 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":3,"thread_start_us":361,"threads_started":1}
I20260806 01:12:00.743161 17187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3b792ecc644efab8bef9519557f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:00.743937 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.727174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.22.63.66:38097)
0806 01:12:00.735127 (+ 7953us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.735135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.735161 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:12:00.735261 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.735270 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.735516 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.735717 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:00.736454 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.737978 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.740345 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.741832 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.742783 (+ 951us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.743314 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.743391 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.743576 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7791,"thread_start_us":7653,"threads_started":1}
I20260806 01:12:00.744648 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.727274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:50938)
0806 01:12:00.733871 (+ 6597us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.733893 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.734019 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.735121 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.735126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.735153 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:00.735530 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.735546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.739868 (+ 4322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.739872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.741939 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.741956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.743780 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.743787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.743961 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.744078 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.744167 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.744297 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6464,"thread_start_us":6313,"threads_started":1}
I20260806 01:12:00.744692 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.727802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50938 (local address 127.22.63.65:45603)
0806 01:12:00.734245 (+ 6443us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.734253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.734282 (+ 29us) server_negotiation.cc:407] Connection header received
0806 01:12:00.734381 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.734389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.734643 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.734831 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:00.735786 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.738378 (+ 2592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.742141 (+ 3763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.743538 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.743688 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.743994 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.744220 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.744373 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6247,"thread_start_us":6273,"threads_started":1}
I20260806 01:12:00.747394 16993 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3b792ecc644efab8bef9519557f725 (DEFAULT_TABLE table=test-table [id=88c3ad1627604f56a59ed95b88674db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:00.748237 16807 tablet_service.cc:1467] Processing CreateTablet for tablet 3c3b792ecc644efab8bef9519557f725 (DEFAULT_TABLE table=test-table [id=88c3ad1627604f56a59ed95b88674db8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:00.748914 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3b792ecc644efab8bef9519557f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:00.749590 16807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c3b792ecc644efab8bef9519557f725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:00.758513 17311 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Bootstrap starting.
I20260806 01:12:00.761754 17312 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Bootstrap starting.
I20260806 01:12:00.762769 17311 tablet_bootstrap.cc:654] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:00.764890 17311 log.cc:824] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:00.765632 17312 tablet_bootstrap.cc:654] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:00.767280 17312 log.cc:824] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:00.767860 17311 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: No bootstrap required, opened a new log
I20260806 01:12:00.768150 17311 ts_tablet_manager.cc:1397] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 01:12:00.768401 17313 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Bootstrap starting.
I20260806 01:12:00.770874 17312 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: No bootstrap required, opened a new log
I20260806 01:12:00.771153 17312 ts_tablet_manager.cc:1397] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 01:12:00.772097 17313 tablet_bootstrap.cc:654] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:00.773360 17313 log.cc:824] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:00.776360 17313 tablet_bootstrap.cc:492] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: No bootstrap required, opened a new log
I20260806 01:12:00.776643 17313 ts_tablet_manager.cc:1397] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 01:12:00.778529 17311 raft_consensus.cc:348] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.778909 17311 raft_consensus.cc:374] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:00.779016 17311 raft_consensus.cc:729] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07aa04f3a4894df89599883f69b32a0f, State: Initialized, Role: FOLLOWER
I20260806 01:12:00.779654 17311 consensus_queue.cc:260] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.780864 17285 heartbeater.cc:503] Master 127.22.63.126:42243 was elected leader, sending a full tablet report...
I20260806 01:12:00.780864 17312 raft_consensus.cc:348] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.781296 17312 raft_consensus.cc:374] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:00.781383 17312 raft_consensus.cc:729] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28a115c430cd4eac889c688527eb42f4, State: Initialized, Role: FOLLOWER
I20260806 01:12:00.781558 17311 ts_tablet_manager.cc:1428] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260806 01:12:00.782079 17312 consensus_queue.cc:260] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.784000 17312 ts_tablet_manager.cc:1428] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260806 01:12:00.784679 17084 heartbeater.cc:503] Master 127.22.63.126:42243 was elected leader, sending a full tablet report...
I20260806 01:12:00.787128 17313 raft_consensus.cc:348] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.787590 17313 raft_consensus.cc:374] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:00.787688 17313 raft_consensus.cc:729] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46edfd6c30a41ea9ff48b67152fd400, State: Initialized, Role: FOLLOWER
I20260806 01:12:00.788291 17313 consensus_queue.cc:260] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.790005 17313 ts_tablet_manager.cc:1428] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260806 01:12:00.791630 17322 raft_consensus.cc:482] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:00.791955 17322 raft_consensus.cc:504] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.793989 17322 leader_election.cc:288] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07aa04f3a4894df89599883f69b32a0f (127.22.63.67:35781), 28a115c430cd4eac889c688527eb42f4 (127.22.63.66:38097)
I20260806 01:12:00.800990 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.794273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:41287 (local address 127.22.63.67:35781)
0806 01:12:00.794536 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.794545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.795085 (+ 540us) server_negotiation.cc:407] Connection header received
0806 01:12:00.795291 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.795297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.795384 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.795475 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:00.796317 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.797360 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.799092 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.799989 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.800188 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.800270 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.800579 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.800712 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 01:12:00.801074 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.794152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:35781 (local address 127.22.63.65:41287)
0806 01:12:00.794971 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.794991 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.795137 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.795653 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.795657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.795681 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:00.796174 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.796185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.797485 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.797489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.798921 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.798934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.800212 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.800218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.800409 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.800433 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.800555 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.800685 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":431,"threads_started":1}
I20260806 01:12:00.801314 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.795229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:49239 (local address 127.22.63.66:38097)
0806 01:12:00.795441 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.795449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.796003 (+ 554us) server_negotiation.cc:407] Connection header received
0806 01:12:00.796226 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.796230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.796330 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.796433 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:00.796862 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.797764 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.799485 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.800355 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.800535 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.800616 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.800916 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.801042 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:12:00.802126 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.794931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:38097 (local address 127.22.63.65:49239)
0806 01:12:00.795897 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:00.795911 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:00.796073 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:00.796435 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:00.796437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:00.796457 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:00.796718 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.796725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.797916 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.797921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.799313 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:00.799324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.800585 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.800591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:00.800748 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.800772 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 01:12:00.800889 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 01:12:00.801014 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":567,"threads_started":1}
I20260806 01:12:00.802487 17215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3b792ecc644efab8bef9519557f725" candidate_uuid: "c46edfd6c30a41ea9ff48b67152fd400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa04f3a4894df89599883f69b32a0f" is_pre_election: true
I20260806 01:12:00.803068 17215 raft_consensus.cc:2454] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c46edfd6c30a41ea9ff48b67152fd400 in term 0.
I20260806 01:12:00.803211 17016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3b792ecc644efab8bef9519557f725" candidate_uuid: "c46edfd6c30a41ea9ff48b67152fd400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a115c430cd4eac889c688527eb42f4" is_pre_election: true
I20260806 01:12:00.803763 17016 raft_consensus.cc:2454] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c46edfd6c30a41ea9ff48b67152fd400 in term 0.
I20260806 01:12:00.804056 16737 leader_election.cc:302] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aa04f3a4894df89599883f69b32a0f, c46edfd6c30a41ea9ff48b67152fd400; no voters:
I20260806 01:12:00.804482 17322 raft_consensus.cc:2781] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:00.804636 17322 raft_consensus.cc:482] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:00.804745 17322 raft_consensus.cc:3037] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:00.807008 17322 raft_consensus.cc:504] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
I20260806 01:12:00.807706 17322 leader_election.cc:288] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [CANDIDATE]: Term 1 election: Requested vote from peers 07aa04f3a4894df89599883f69b32a0f (127.22.63.67:35781), 28a115c430cd4eac889c688527eb42f4 (127.22.63.66:38097)
I20260806 01:12:00.808171 17215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3b792ecc644efab8bef9519557f725" candidate_uuid: "c46edfd6c30a41ea9ff48b67152fd400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07aa04f3a4894df89599883f69b32a0f"
I20260806 01:12:00.808285 17016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c3b792ecc644efab8bef9519557f725" candidate_uuid: "c46edfd6c30a41ea9ff48b67152fd400" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28a115c430cd4eac889c688527eb42f4"
I20260806 01:12:00.808393 17215 raft_consensus.cc:3037] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:00.808512 17016 raft_consensus.cc:3037] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:00.810725 17215 raft_consensus.cc:2454] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c46edfd6c30a41ea9ff48b67152fd400 in term 1.
I20260806 01:12:00.811156 16737 leader_election.cc:302] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07aa04f3a4894df89599883f69b32a0f, c46edfd6c30a41ea9ff48b67152fd400; no voters:
I20260806 01:12:00.811470 17322 raft_consensus.cc:2781] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:00.811765 17016 raft_consensus.cc:2454] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c46edfd6c30a41ea9ff48b67152fd400 in term 1.
I20260806 01:12:00.812124 17322 raft_consensus.cc:686] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 1 LEADER]: Becoming Leader. State: Replica: c46edfd6c30a41ea9ff48b67152fd400, State: Running, Role: LEADER
I20260806 01:12:00.812690 17322 consensus_queue.cc:237] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } }
W20260806 01:12:00.816812 17086 tablet.cc:2369] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:00.817672 16611 catalog_manager.cc:5495] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 reported cstate change: term changed from 0 to 1, leader changed from <none> to c46edfd6c30a41ea9ff48b67152fd400 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "c46edfd6c30a41ea9ff48b67152fd400" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c46edfd6c30a41ea9ff48b67152fd400" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 45603 } health_report { overall_health: HEALTHY } } }
I20260806 01:12:00.836230 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.830805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.22.63.65:45603)
0806 01:12:00.831031 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.831039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.831058 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:12:00.831154 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.831158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.831232 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.831386 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:00.831814 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.832549 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.834002 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.834423 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.834558 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.835102 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:00.835600 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:00.835797 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.835885 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.835964 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:12:00.844216 17215 raft_consensus.cc:1261] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 1 FOLLOWER]: Refusing update from remote peer c46edfd6c30a41ea9ff48b67152fd400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:00.844375 17016 raft_consensus.cc:1261] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 1 FOLLOWER]: Refusing update from remote peer c46edfd6c30a41ea9ff48b67152fd400: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:00.845176 17322 consensus_queue.cc:1035] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07aa04f3a4894df89599883f69b32a0f" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:00.845566 17327 consensus_queue.cc:1035] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28a115c430cd4eac889c688527eb42f4" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 38097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:00.858862 17337 mvcc.cc:205] Tried to move back new op lower bound from 7315368840567881728 to 7315368840452190208. Current Snapshot: MvccSnapshot[applied={T|T < 7315368840567881728}]
I20260806 01:12:00.871021 16612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:00.871320 16612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:00.874770 16612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48785e8e78b64d88af56ad80839d0904: Sending DeleteTablet for 3 replicas of tablet 3c3b792ecc644efab8bef9519557f725
I20260806 01:12:00.875972 17187 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c3b792ecc644efab8bef9519557f725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:00 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:51222
I20260806 01:12:00.876061 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c3b792ecc644efab8bef9519557f725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:00 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:57324
I20260806 01:12:00.876150 16807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c3b792ecc644efab8bef9519557f725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:00 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:50938
I20260806 01:12:00.878557 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 16710
I20260806 01:12:00.888373 17354 tablet_replica.cc:330] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: stopping tablet replica
I20260806 01:12:00.888607 17353 tablet_replica.cc:330] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: stopping tablet replica
I20260806 01:12:00.889106 17354 raft_consensus.cc:2229] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:00.889132 17353 raft_consensus.cc:2229] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:00.889703 17353 raft_consensus.cc:2258] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:00.889906 17354 raft_consensus.cc:2258] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:00.890731 17352 tablet_replica.cc:330] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: stopping tablet replica
I20260806 01:12:00.891171 17352 raft_consensus.cc:2229] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:00.891670 17352 raft_consensus.cc:2258] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:00.893875 17354 ts_tablet_manager.cc:1905] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:00.894014 17352 ts_tablet_manager.cc:1905] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:00.896754 17353 ts_tablet_manager.cc:1905] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:12:00 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:12:00.902110 17352 ts_tablet_manager.cc:1918] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:00.902302 17352 log.cc:1196] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/3c3b792ecc644efab8bef9519557f725
I20260806 01:12:00.902956 17352 ts_tablet_manager.cc:1939] T 3c3b792ecc644efab8bef9519557f725 P 07aa04f3a4894df89599883f69b32a0f: Deleting consensus metadata
I20260806 01:12:00.905155 16591 catalog_manager.cc:4841] TS 07aa04f3a4894df89599883f69b32a0f (127.22.63.67:35781): tablet 3c3b792ecc644efab8bef9519557f725 (table test-table [id=88c3ad1627604f56a59ed95b88674db8]) successfully deleted
I20260806 01:12:00.905871 17353 ts_tablet_manager.cc:1918] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:00.906062 17353 log.cc:1196] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/3c3b792ecc644efab8bef9519557f725
I20260806 01:12:00.906677 17353 ts_tablet_manager.cc:1939] T 3c3b792ecc644efab8bef9519557f725 P 28a115c430cd4eac889c688527eb42f4: Deleting consensus metadata
I20260806 01:12:00.908644 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:00.880305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.22.63.65:45603)
0806 01:12:00.880497 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:00.880508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:00.884695 (+ 4187us) server_negotiation.cc:407] Connection header received
0806 01:12:00.885745 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:00.885753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:00.885886 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:00.885979 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:00.890360 (+ 4381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.891205 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.893254 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:00.893923 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:00.894028 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:00.900062 (+ 6034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:00.900076 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:00.900100 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:00.900161 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:00.904580 (+ 4419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.905603 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.905616 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.905622 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.905922 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:00.906499 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:00.906504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:00.906507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:00.907012 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:00.907309 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:00.907941 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 01:12:00.908212 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:12:00.908699 16591 catalog_manager.cc:4841] TS 28a115c430cd4eac889c688527eb42f4 (127.22.63.66:38097): tablet 3c3b792ecc644efab8bef9519557f725 (table test-table [id=88c3ad1627604f56a59ed95b88674db8]) successfully deleted
I20260806 01:12:00.910348 16887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:01.007306 17354 ts_tablet_manager.cc:1918] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:01.007601 17354 log.cc:1196] T 3c3b792ecc644efab8bef9519557f725 P c46edfd6c30a41ea9ff48b67152fd400: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/3c3b792ecc644efab8bef9519557f725
I20260806 01:12:01.008075 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 16710
I20260806 01:12:01.035668 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 16902
W20260806 01:12:01.035668 16591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:45603 (error 108)
I20260806 01:12:01.039108 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.038714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:50974)
0806 01:12:01.038868 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.038970 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 01:12:01.039458 16591 catalog_manager.cc:4584] TS c46edfd6c30a41ea9ff48b67152fd400 (127.22.63.65:45603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c3b792ecc644efab8bef9519557f725: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Aug 06 01:12:01 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:12:01.051996 17306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.036586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.22.63.66:38097)
0806 01:12:01.036817 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:01.036825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:01.036926 (+ 101us) server_negotiation.cc:407] Connection header received
0806 01:12:01.037130 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:01.037138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:01.037224 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:01.037328 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:01.038694 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:01.039654 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:01.041188 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:01.041567 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:01.041663 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:01.046021 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:01.046034 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:01.046049 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:01.046091 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:01.049170 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:01.049862 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:01.049869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:01.049871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:01.050003 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:01.050471 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:01.050474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:01.050476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:01.050903 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:01.051122 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:01.051395 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 01:12:01.051552 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:12:01.052799 17079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:01.093420 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.092892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:50980)
0806 01:12:01.093132 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.093241 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 01:12:01.094525 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.094173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:50988)
0806 01:12:01.094329 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.094410 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 01:12:01.144275 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 16902
I20260806 01:12:01.155023 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.154533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:51002)
0806 01:12:01.154765 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.154852 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:12:01.156040 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.155683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:51004)
0806 01:12:01.155861 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.155927 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:12:01.173494 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 17094
Aug 06 01:12:01 dist-test-slave-7f01 krb5kdc[16546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:12:01.189121 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.174281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.22.63.67:35781)
0806 01:12:01.174454 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:01.174458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:01.174477 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:12:01.174638 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:01.174642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:01.174717 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:01.174807 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:01.176222 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:01.176946 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:01.178304 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:01.178709 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:01.178805 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:01.183368 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:01.183382 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:01.183399 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:01.183436 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:01.186267 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:01.186967 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:01.186982 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:01.186985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:01.187128 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:01.187513 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:01.187518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:01.187521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:01.187944 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:01.188249 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:01.188558 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 01:12:01.188785 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:12:01.189824 17277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:01.236588 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.236020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:51016)
0806 01:12:01.236257 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.236418 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 01:12:01.237876 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.237339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:51020)
0806 01:12:01.237616 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.237732 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260806 01:12:01.283193 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 17094
I20260806 01:12:01.308759 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 16562
I20260806 01:12:01.309993 16658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:01.422040 17308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:01.421299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:45603 (local address 127.0.0.1:51034)
0806 01:12:01.421677 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:01.421835 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260806 01:12:01.422142 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 16562
2026-08-06T01:12:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4785 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T01:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:01Z Disabled control of system clock
I20260806 01:12:01.524005 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37099
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:02.106740 17369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:02.107380 17369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:02.107721 17369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:02.122097 17369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:02.122375 17369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:02.122561 17369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:02.122767 17369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:02.122921 17369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:02.140720 17369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:02.143649 17369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:02.147641 17369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:02.170720 17378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:02.170604 17377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:02.170753 17381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:02.174499 17369 server_base.cc:1034] running on GCE node
I20260806 01:12:02.176118 17369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:02.178927 17369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:02.180243 17369 hybrid_clock.cc:648] HybridClock initialized: now 1785978722180162 us; error 47 us; skew 500 ppm
I20260806 01:12:02.180855 17369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:02.185323 17369 webserver.cc:460] Webserver started at http://127.22.63.126:40113/ using document root <none> and password file <none>
I20260806 01:12:02.186582 17369 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:02.186811 17369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:02.187472 17369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:02.193070 17369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "4ec5287d13504203a84822b7481b6558"
format_stamp: "Formatted at 2026-08-06 01:12:02 on dist-test-slave-7f01"
server_key: "9310a3979ef5d92ee16e05cc556f93b1"
server_key_iv: "281356b2bc6adbb83f7579887cc75612"
server_key_version: "encryptionkey@0"
I20260806 01:12:02.194823 17369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "4ec5287d13504203a84822b7481b6558"
format_stamp: "Formatted at 2026-08-06 01:12:02 on dist-test-slave-7f01"
server_key: "9310a3979ef5d92ee16e05cc556f93b1"
server_key_iv: "281356b2bc6adbb83f7579887cc75612"
server_key_version: "encryptionkey@0"
I20260806 01:12:02.205139 17369 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.011s sys 0.000s
I20260806 01:12:02.212172 17391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:02.214834 17369 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260806 01:12:02.215232 17369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "4ec5287d13504203a84822b7481b6558"
format_stamp: "Formatted at 2026-08-06 01:12:02 on dist-test-slave-7f01"
server_key: "9310a3979ef5d92ee16e05cc556f93b1"
server_key_iv: "281356b2bc6adbb83f7579887cc75612"
server_key_version: "encryptionkey@0"
I20260806 01:12:02.215683 17369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:02.252023 17369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:02.253757 17369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:02.254326 17369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:02.290200 17369 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37099
I20260806 01:12:02.290261 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37099 every 8 connection(s)
I20260806 01:12:02.292897 17369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:02.300071 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 17369
I20260806 01:12:02.300480 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:02.301007 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:02.301709 22781 external_mini_cluster.cc:1428] Setting key b93a89bdb4dff304cb442fe67f45b99b
I20260806 01:12:02.329531 17471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: Bootstrap starting.
I20260806 01:12:02.331806 17475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:02.305584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.22.63.126:37099)
0806 01:12:02.308188 (+ 2604us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:02.308212 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:02.308332 (+ 120us) server_negotiation.cc:407] Connection header received
0806 01:12:02.311380 (+ 3048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:02.311422 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:02.312486 (+ 1064us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:02.313096 (+ 610us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:02.314574 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:02.317002 (+ 2428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:02.321007 (+ 4005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:02.322610 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:02.322889 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:02.325566 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:02.325582 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:02.325628 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:02.327009 (+ 1381us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:02.327246 (+ 237us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:02.327286 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:02.327479 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:02.328026 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 01:12:02.328801 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1761,"thread_start_us":1159,"threads_started":1}
I20260806 01:12:02.336476 17471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:02.338653 17471 log.cc:824] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:02.345192 17471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: No bootstrap required, opened a new log
I20260806 01:12:02.364027 17471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } }
I20260806 01:12:02.364907 17471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:02.365185 17471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec5287d13504203a84822b7481b6558, State: Initialized, Role: FOLLOWER
I20260806 01:12:02.366386 17471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } }
I20260806 01:12:02.366868 17471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:02.367163 17471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:02.367501 17471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:02.372239 17471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } }
I20260806 01:12:02.373204 17471 leader_election.cc:302] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ec5287d13504203a84822b7481b6558; no voters:
I20260806 01:12:02.374636 17471 leader_election.cc:288] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:02.374894 17477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:02.376354 17477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec5287d13504203a84822b7481b6558, State: Running, Role: LEADER
I20260806 01:12:02.377321 17477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } }
I20260806 01:12:02.378933 17471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:02.383100 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ec5287d13504203a84822b7481b6558. Latest consensus state: current_term: 1 leader_uuid: "4ec5287d13504203a84822b7481b6558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } } }
I20260806 01:12:02.383945 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:02.383288 17478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ec5287d13504203a84822b7481b6558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ec5287d13504203a84822b7481b6558" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37099 } } }
I20260806 01:12:02.384196 17478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:02.391063 17485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:02.405146 17485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:02.424103 17485 catalog_manager.cc:1347] Generated new cluster ID: 0c86acf9f04346efa669767083ae4b27
I20260806 01:12:02.424365 17485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:02.447578 17485 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:02.449939 17485 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:02.494221 17485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: Generated new TSK 0
I20260806 01:12:02.505844 17485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:02.534277 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--builtin_ntp_servers=127.22.63.84:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:03.248203 17507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:03.248821 17507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:03.249142 17507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:03.268625 17507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:03.269224 17507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:03.269910 17507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:12:03.309424 17507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:03.313793 17507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:03.320863 17507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:03.403856 17521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:03.403899 17519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:03.404865 17517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:03.410269 17507 server_base.cc:1034] running on GCE node
I20260806 01:12:03.411772 17507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:03.414064 17507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:03.417637 17507 hybrid_clock.cc:648] HybridClock initialized: now 1785978723417506 us; error 86 us; skew 500 ppm
I20260806 01:12:03.418663 17507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:03.424227 17507 webserver.cc:460] Webserver started at http://127.22.63.65:36513/ using document root <none> and password file <none>
I20260806 01:12:03.425971 17507 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:03.426298 17507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:03.427026 17507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:03.434456 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "44b51cd5df8a44a2ab86e39e48f435bb"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "33ddc7790a5ffd6ded0e52145ecd5489"
server_key_iv: "974b35d646f4d46311d68ab359645ade"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.436534 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "44b51cd5df8a44a2ab86e39e48f435bb"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "33ddc7790a5ffd6ded0e52145ecd5489"
server_key_iv: "974b35d646f4d46311d68ab359645ade"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.449141 17507 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.010s sys 0.001s
I20260806 01:12:03.457927 17529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:03.460641 17507 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.002s
I20260806 01:12:03.460974 17507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "44b51cd5df8a44a2ab86e39e48f435bb"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "33ddc7790a5ffd6ded0e52145ecd5489"
server_key_iv: "974b35d646f4d46311d68ab359645ade"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.461445 17507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:03.486919 17507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:03.488596 17507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:03.489214 17507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:03.491075 17507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:03.493548 17507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:03.493680 17507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:03.493832 17507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:03.493925 17507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:03.532327 17507 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:37449
I20260806 01:12:03.532387 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:37449 every 8 connection(s)
I20260806 01:12:03.533768 17507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:12:03.537863 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 17507
I20260806 01:12:03.538077 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:12:03.538582 22781 external_mini_cluster.cc:1428] Setting key 19f7ed532075d747c724783e74e77ea3
I20260806 01:12:03.541853 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--builtin_ntp_servers=127.22.63.84:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:03.549337 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:03.536602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:56135 (local address 127.22.63.126:37099)
0806 01:12:03.537115 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:03.537119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:03.538305 (+ 1186us) server_negotiation.cc:407] Connection header received
0806 01:12:03.540642 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:03.540646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:03.540749 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:03.540876 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:03.541968 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:03.543480 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:03.545431 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:03.546297 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:03.546475 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:03.548228 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:03.548238 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:03.548245 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:03.548277 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:03.548330 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:03.548379 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:03.548487 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:03.548729 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 01:12:03.548852 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":295,"threads_started":1}
I20260806 01:12:03.550186 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:03.537044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37099 (local address 127.22.63.65:56135)
0806 01:12:03.538106 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:03.538173 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:03.540308 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:03.541093 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:03.541107 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:03.541197 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:03.541774 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:03.541790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:03.543902 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:03.543907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:03.545252 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:03.545265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:03.546582 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:03.546588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:03.546855 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:03.547817 (+ 962us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:03.547821 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:03.547867 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:03.547974 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:03.548500 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:03.548509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:03.548516 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:12:03.548670 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 01:12:03.549020 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":361,"threads_started":1}
I20260806 01:12:03.552034 17689 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37099
I20260806 01:12:03.552420 17689 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:03.553263 17689 heartbeater.cc:511] Master 127.22.63.126:37099 requested a full tablet report, sending...
I20260806 01:12:03.555557 17419 ts_manager.cc:194] Registered new tserver with Master: 44b51cd5df8a44a2ab86e39e48f435bb (127.22.63.65:37449)
I20260806 01:12:03.557286 17419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.65:56135
W20260806 01:12:03.917065 17694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:03.917403 17694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:03.917562 17694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:03.924952 17694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:03.925061 17694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:03.925220 17694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:12:03.934779 17694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:03.936167 17694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:03.937978 17694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:03.952390 17703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:03.952538 17704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:03.952558 17707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:03.959702 17694 server_base.cc:1034] running on GCE node
I20260806 01:12:03.960503 17694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:03.961570 17694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:03.962836 17694 hybrid_clock.cc:648] HybridClock initialized: now 1785978723962767 us; error 51 us; skew 500 ppm
I20260806 01:12:03.963193 17694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:03.968596 17694 webserver.cc:460] Webserver started at http://127.22.63.66:41329/ using document root <none> and password file <none>
I20260806 01:12:03.969422 17694 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:03.969573 17694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:03.969918 17694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:03.973186 17694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "4c4c28bbb0c6dbb83a2b9b011d894503"
server_key_iv: "e363a6c160a9a84d6e82f076d8d428fa"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.974028 17694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "4c4c28bbb0c6dbb83a2b9b011d894503"
server_key_iv: "e363a6c160a9a84d6e82f076d8d428fa"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.979521 17694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:12:03.983217 17715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:03.984810 17694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:03.985035 17694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd"
format_stamp: "Formatted at 2026-08-06 01:12:03 on dist-test-slave-7f01"
server_key: "4c4c28bbb0c6dbb83a2b9b011d894503"
server_key_iv: "e363a6c160a9a84d6e82f076d8d428fa"
server_key_version: "encryptionkey@0"
I20260806 01:12:03.985280 17694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:04.001483 17694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:04.002336 17694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:04.002664 17694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:04.003674 17694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:04.005486 17694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:04.005617 17694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:04.005757 17694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:04.005869 17694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:04.044031 17694 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:43215
I20260806 01:12:04.044098 17881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:43215 every 8 connection(s)
I20260806 01:12:04.045617 17694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:04.050203 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 17694
I20260806 01:12:04.050371 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:04.051165 22781 external_mini_cluster.cc:1428] Setting key 666602919aecf1921001b12b37a36f29
I20260806 01:12:04.054090 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--builtin_ntp_servers=127.22.63.84:38339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:04.061704 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.049435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:43253 (local address 127.22.63.126:37099)
0806 01:12:04.049684 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.049690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.052191 (+ 2501us) server_negotiation.cc:407] Connection header received
0806 01:12:04.054116 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.054122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.054209 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.054329 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:04.055486 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.056691 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.058323 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.059047 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.059193 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.060694 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:04.060703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:04.060711 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:04.060738 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:04.060785 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:04.060803 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:04.060908 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.061180 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.061297 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 01:12:04.062651 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.050117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37099 (local address 127.22.63.66:43253)
0806 01:12:04.052006 (+ 1889us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.052064 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.053828 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.054609 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.054618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.054713 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:04.055310 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.055326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.056870 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.056874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.058169 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.058181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.059281 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.059286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.059516 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.060355 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:04.060358 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:04.060402 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:04.060486 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:04.060942 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:04.060952 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:04.060959 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.061139 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.061458 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":611,"threads_started":1}
I20260806 01:12:04.064283 17882 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37099
I20260806 01:12:04.064630 17882 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:04.065622 17882 heartbeater.cc:511] Master 127.22.63.126:37099 requested a full tablet report, sending...
I20260806 01:12:04.067333 17419 ts_manager.cc:194] Registered new tserver with Master: 4967a29a76ba4b91a1ce8cf6d3bed7bd (127.22.63.66:43215)
I20260806 01:12:04.068225 17419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.66:43253
W20260806 01:12:04.424314 17886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:04.424608 17886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:04.424696 17886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:04.432415 17886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:04.432526 17886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:04.432636 17886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:04.442308 17886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:37099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:04.443578 17886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:04.445412 17886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:04.459642 17896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:04.459684 17893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:04.459596 17892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:04.461163 17886 server_base.cc:1034] running on GCE node
I20260806 01:12:04.461829 17886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:04.462847 17886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:04.464064 17886 hybrid_clock.cc:648] HybridClock initialized: now 1785978724464020 us; error 32 us; skew 500 ppm
I20260806 01:12:04.464447 17886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:04.467283 17886 webserver.cc:460] Webserver started at http://127.22.63.67:39301/ using document root <none> and password file <none>
I20260806 01:12:04.468194 17886 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:04.468335 17886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:04.468739 17886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:04.472388 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "b02c3ad0551946beb01aad3f2d4a5931"
format_stamp: "Formatted at 2026-08-06 01:12:04 on dist-test-slave-7f01"
server_key: "2103dfec428c9fb81c882a0986ce6107"
server_key_iv: "5ad5d3422902dddf902c0683e3b23227"
server_key_version: "encryptionkey@0"
I20260806 01:12:04.473335 17886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "b02c3ad0551946beb01aad3f2d4a5931"
format_stamp: "Formatted at 2026-08-06 01:12:04 on dist-test-slave-7f01"
server_key: "2103dfec428c9fb81c882a0986ce6107"
server_key_iv: "5ad5d3422902dddf902c0683e3b23227"
server_key_version: "encryptionkey@0"
I20260806 01:12:04.479557 17886 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260806 01:12:04.483623 17904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:04.485164 17886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:04.485369 17886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "b02c3ad0551946beb01aad3f2d4a5931"
format_stamp: "Formatted at 2026-08-06 01:12:04 on dist-test-slave-7f01"
server_key: "2103dfec428c9fb81c882a0986ce6107"
server_key_iv: "5ad5d3422902dddf902c0683e3b23227"
server_key_version: "encryptionkey@0"
I20260806 01:12:04.485642 17886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:04.502135 17886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:04.503147 17886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:04.503496 17886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:04.504773 17886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:04.506549 17886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:04.506690 17886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:04.506848 17886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:04.506939 17886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:04.547686 17886 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:44769
I20260806 01:12:04.547717 18065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:44769 every 8 connection(s)
I20260806 01:12:04.549156 17886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:04.551180 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 17886
I20260806 01:12:04.551378 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:04.551920 22781 external_mini_cluster.cc:1428] Setting key 0b29f5c668a6b59236a20023ace44b2d
I20260806 01:12:04.560745 17689 heartbeater.cc:503] Master 127.22.63.126:37099 was elected leader, sending a full tablet report...
I20260806 01:12:04.566371 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.552153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:37335 (local address 127.22.63.126:37099)
0806 01:12:04.552393 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.552401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.554248 (+ 1847us) server_negotiation.cc:407] Connection header received
0806 01:12:04.556376 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.556381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.556456 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.556550 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:04.557624 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.558393 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.560097 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.560662 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.560805 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.565040 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:04.565045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:04.565049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:04.565069 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:04.565108 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:04.565121 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:04.565205 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.565546 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.565649 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:12:04.567481 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.552891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37099 (local address 127.22.63.67:37335)
0806 01:12:04.554065 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.554129 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.556121 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.556766 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.556774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.556863 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:04.557458 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.557477 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.558551 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.558555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.559949 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.559962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.560943 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.560954 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.561187 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.564690 (+ 3503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:04.564694 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:04.564741 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:04.564835 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:04.565220 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:04.565228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:04.565235 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.565454 (+ 219us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.565874 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":380,"threads_started":1}
I20260806 01:12:04.569381 18066 heartbeater.cc:347] Connected to a master server at 127.22.63.126:37099
I20260806 01:12:04.569736 18066 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:04.570681 18066 heartbeater.cc:511] Master 127.22.63.126:37099 requested a full tablet report, sending...
I20260806 01:12:04.572404 17417 ts_manager.cc:194] Registered new tserver with Master: b02c3ad0551946beb01aad3f2d4a5931 (127.22.63.67:44769)
I20260806 01:12:04.573408 17417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.67:37335
I20260806 01:12:04.581972 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:04.593617 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.586695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53586 (local address 127.22.63.126:37099)
0806 01:12:04.586993 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.587001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.587309 (+ 308us) server_negotiation.cc:407] Connection header received
0806 01:12:04.587540 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.587544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.587624 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.587739 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:04.588313 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.589042 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.590624 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.591272 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.591415 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.592716 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:04.592724 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:04.592734 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:04.592754 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:04.592787 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:04.592798 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:04.592897 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.593054 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.593156 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260806 01:12:04.598455 17417 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:04.606144 17417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:04.641139 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.620935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:37449 (local address 127.0.0.1:60018)
0806 01:12:04.622950 (+ 2015us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.622966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.623059 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.632503 (+ 9444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.632512 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.632554 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:04.633069 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.633091 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.635502 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.635508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.638289 (+ 2781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.638304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.640164 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.640172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.640399 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.640524 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.640654 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.640830 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"mutex_wait_us":55,"thread_start_us":13676,"threads_started":1}
I20260806 01:12:04.642320 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.622506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.22.63.65:37449)
0806 01:12:04.631667 (+ 9161us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.631675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.631706 (+ 31us) server_negotiation.cc:407] Connection header received
0806 01:12:04.631873 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.631882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.632203 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.633355 (+ 1152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:04.633691 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.635326 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.638506 (+ 3180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.639943 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.640089 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.640407 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.641742 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.641928 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8985,"thread_start_us":8844,"threads_started":1}
I20260806 01:12:04.643221 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.620345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:43215 (local address 127.0.0.1:58280)
0806 01:12:04.621877 (+ 1532us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.621916 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.622111 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.635981 (+ 13870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.635987 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.636016 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:04.636773 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.636790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.638642 (+ 1852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.638647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.640474 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.640492 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.642370 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.642377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.642551 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.642584 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.642678 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.642812 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1287,"mutex_wait_us":112,"thread_start_us":1148,"threads_started":1}
I20260806 01:12:04.643311 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.620593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.22.63.66:43215)
0806 01:12:04.635208 (+ 14615us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.635217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.635249 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:12:04.635360 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.635371 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.635661 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.636015 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:04.636992 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.638499 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.640690 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.642160 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.642308 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.642569 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.642809 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.642978 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14459,"thread_start_us":14326,"threads_started":1}
I20260806 01:12:04.643918 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.622350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:44769 (local address 127.0.0.1:55170)
0806 01:12:04.634409 (+ 12059us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.634429 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.634546 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.636399 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.636404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.636434 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:04.637227 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.637243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.639192 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.639197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.641468 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.641486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.643133 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.643142 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.643357 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.643387 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.643463 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.643595 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11918,"thread_start_us":349,"threads_started":1}
I20260806 01:12:04.645183 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.623002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.22.63.67:44769)
0806 01:12:04.635778 (+ 12776us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.635790 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.635828 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:12:04.635944 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.635951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.636171 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.636393 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:04.637443 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.639022 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.641685 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.642908 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.644267 (+ 1359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.644562 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.644644 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.644806 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12562,"thread_start_us":12394,"threads_started":1}
I20260806 01:12:04.646917 17790 tablet_service.cc:1467] Processing CreateTablet for tablet 49f9adad798f4336b953a1f27bb9a78b (DEFAULT_TABLE table=test-table [id=e63b176bf54a40e4b776a64678274fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:04.646909 17601 tablet_service.cc:1467] Processing CreateTablet for tablet 49f9adad798f4336b953a1f27bb9a78b (DEFAULT_TABLE table=test-table [id=e63b176bf54a40e4b776a64678274fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:04.648552 17601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f9adad798f4336b953a1f27bb9a78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:04.648653 17790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f9adad798f4336b953a1f27bb9a78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:04.648355 17973 tablet_service.cc:1467] Processing CreateTablet for tablet 49f9adad798f4336b953a1f27bb9a78b (DEFAULT_TABLE table=test-table [id=e63b176bf54a40e4b776a64678274fa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:04.649598 17973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f9adad798f4336b953a1f27bb9a78b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:04.662370 18092 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Bootstrap starting.
I20260806 01:12:04.665239 18097 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Bootstrap starting.
I20260806 01:12:04.666091 18092 tablet_bootstrap.cc:654] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:04.666113 18093 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Bootstrap starting.
I20260806 01:12:04.667565 18092 log.cc:824] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:04.669807 18097 tablet_bootstrap.cc:654] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:04.669867 18093 tablet_bootstrap.cc:654] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:04.670931 18097 log.cc:824] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:04.671195 18092 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: No bootstrap required, opened a new log
I20260806 01:12:04.671211 18093 log.cc:824] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:04.671499 18092 ts_tablet_manager.cc:1397] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 01:12:04.674214 18097 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: No bootstrap required, opened a new log
I20260806 01:12:04.674429 18093 tablet_bootstrap.cc:492] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: No bootstrap required, opened a new log
I20260806 01:12:04.674503 18097 ts_tablet_manager.cc:1397] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260806 01:12:04.674710 18093 ts_tablet_manager.cc:1397] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 01:12:04.682457 18093 raft_consensus.cc:348] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.682899 18093 raft_consensus.cc:374] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:04.682612 18092 raft_consensus.cc:348] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.683063 18093 raft_consensus.cc:729] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b51cd5df8a44a2ab86e39e48f435bb, State: Initialized, Role: FOLLOWER
I20260806 01:12:04.683139 18092 raft_consensus.cc:374] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:04.683265 18092 raft_consensus.cc:729] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02c3ad0551946beb01aad3f2d4a5931, State: Initialized, Role: FOLLOWER
I20260806 01:12:04.683985 18093 consensus_queue.cc:260] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.683974 18092 consensus_queue.cc:260] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.685870 18097 raft_consensus.cc:348] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.686476 18097 raft_consensus.cc:374] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:04.686605 18097 raft_consensus.cc:729] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4967a29a76ba4b91a1ce8cf6d3bed7bd, State: Initialized, Role: FOLLOWER
I20260806 01:12:04.687379 18097 consensus_queue.cc:260] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.688808 17882 heartbeater.cc:503] Master 127.22.63.126:37099 was elected leader, sending a full tablet report...
I20260806 01:12:04.689813 18097 ts_tablet_manager.cc:1428] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260806 01:12:04.690721 18093 ts_tablet_manager.cc:1428] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Time spent starting tablet: real 0.016s user 0.007s sys 0.005s
I20260806 01:12:04.691342 18066 heartbeater.cc:503] Master 127.22.63.126:37099 was elected leader, sending a full tablet report...
I20260806 01:12:04.692140 18092 ts_tablet_manager.cc:1428] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Time spent starting tablet: real 0.020s user 0.020s sys 0.000s
I20260806 01:12:04.701182 18107 raft_consensus.cc:482] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:04.701428 18107 raft_consensus.cc:504] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.703655 18107 leader_election.cc:288] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4967a29a76ba4b91a1ce8cf6d3bed7bd (127.22.63.66:43215), 44b51cd5df8a44a2ab86e39e48f435bb (127.22.63.65:37449)
I20260806 01:12:04.711064 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.703879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:43215 (local address 127.22.63.67:42709)
0806 01:12:04.704223 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.704245 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.704424 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.704969 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.704973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.704999 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:04.705195 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.705203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.706605 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.706612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.708173 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.708186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.710149 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.710154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.710371 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.710408 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.710510 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.710666 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":65}
I20260806 01:12:04.711686 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.703928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:42709 (local address 127.22.63.66:43215)
0806 01:12:04.704135 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.704141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.704339 (+ 198us) server_negotiation.cc:407] Connection header received
0806 01:12:04.704628 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.704637 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.704781 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.704918 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:04.705381 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.706473 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.708335 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.709736 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.709932 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.710034 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.710653 (+ 619us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.710887 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:12:04.711712 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.703965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:37729 (local address 127.22.63.65:37449)
0806 01:12:04.704166 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.704172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.705236 (+ 1064us) server_negotiation.cc:407] Connection header received
0806 01:12:04.705439 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.705444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.705568 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.705659 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:04.706343 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.707071 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.709287 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.710502 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.710644 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.710730 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.711356 (+ 626us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.711491 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:12:04.712018 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.704128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:37449 (local address 127.22.63.67:37729)
0806 01:12:04.705120 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:04.705137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:04.705291 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:04.705743 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:04.705748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:04.705776 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:04.706146 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.706157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.707250 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.707256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.709051 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:04.709067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.710899 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.710905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:04.711098 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.711142 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:12:04.711288 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 01:12:04.711447 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":3,"thread_start_us":424,"threads_started":1}
I20260806 01:12:04.713344 17819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f9adad798f4336b953a1f27bb9a78b" candidate_uuid: "b02c3ad0551946beb01aad3f2d4a5931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" is_pre_election: true
I20260806 01:12:04.713332 17629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f9adad798f4336b953a1f27bb9a78b" candidate_uuid: "b02c3ad0551946beb01aad3f2d4a5931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" is_pre_election: true
I20260806 01:12:04.713953 17629 raft_consensus.cc:2454] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02c3ad0551946beb01aad3f2d4a5931 in term 0.
I20260806 01:12:04.713953 17819 raft_consensus.cc:2454] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b02c3ad0551946beb01aad3f2d4a5931 in term 0.
I20260806 01:12:04.714917 17908 leader_election.cc:302] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b51cd5df8a44a2ab86e39e48f435bb, b02c3ad0551946beb01aad3f2d4a5931; no voters:
I20260806 01:12:04.715377 18107 raft_consensus.cc:2781] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:04.715538 18107 raft_consensus.cc:482] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:04.715660 18107 raft_consensus.cc:3037] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:04.717942 18107 raft_consensus.cc:504] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.718645 18107 leader_election.cc:288] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [CANDIDATE]: Term 1 election: Requested vote from peers 4967a29a76ba4b91a1ce8cf6d3bed7bd (127.22.63.66:43215), 44b51cd5df8a44a2ab86e39e48f435bb (127.22.63.65:37449)
I20260806 01:12:04.720736 17629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f9adad798f4336b953a1f27bb9a78b" candidate_uuid: "b02c3ad0551946beb01aad3f2d4a5931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b51cd5df8a44a2ab86e39e48f435bb"
I20260806 01:12:04.720993 17629 raft_consensus.cc:3037] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:04.721299 17819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f9adad798f4336b953a1f27bb9a78b" candidate_uuid: "b02c3ad0551946beb01aad3f2d4a5931" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd"
I20260806 01:12:04.721532 17819 raft_consensus.cc:3037] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:04.724493 17629 raft_consensus.cc:2454] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02c3ad0551946beb01aad3f2d4a5931 in term 1.
I20260806 01:12:04.724949 17819 raft_consensus.cc:2454] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b02c3ad0551946beb01aad3f2d4a5931 in term 1.
I20260806 01:12:04.725021 17908 leader_election.cc:302] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b51cd5df8a44a2ab86e39e48f435bb, b02c3ad0551946beb01aad3f2d4a5931; no voters:
I20260806 01:12:04.725764 18107 raft_consensus.cc:2781] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:04.726354 18107 raft_consensus.cc:686] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 1 LEADER]: Becoming Leader. State: Replica: b02c3ad0551946beb01aad3f2d4a5931, State: Running, Role: LEADER
I20260806 01:12:04.726995 18107 consensus_queue.cc:237] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } }
I20260806 01:12:04.733095 17417 catalog_manager.cc:5495] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 reported cstate change: term changed from 0 to 1, leader changed from <none> to b02c3ad0551946beb01aad3f2d4a5931 (127.22.63.67). New cstate: current_term: 1 leader_uuid: "b02c3ad0551946beb01aad3f2d4a5931" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b02c3ad0551946beb01aad3f2d4a5931" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 44769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 } health_report { overall_health: UNKNOWN } } }
I20260806 01:12:04.767093 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.759285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.22.63.67:44769)
0806 01:12:04.759502 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.759508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.759526 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:12:04.759634 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.759639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.759776 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.759897 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:04.760508 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.761622 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.763876 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.764537 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.764664 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.765346 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:04.766414 (+ 1068us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:04.766564 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.766704 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.766811 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:12:04.777626 17819 raft_consensus.cc:1261] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 1 FOLLOWER]: Refusing update from remote peer b02c3ad0551946beb01aad3f2d4a5931: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:04.777840 17629 raft_consensus.cc:1261] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 1 FOLLOWER]: Refusing update from remote peer b02c3ad0551946beb01aad3f2d4a5931: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:04.778782 18107 consensus_queue.cc:1035] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b51cd5df8a44a2ab86e39e48f435bb" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 37449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:04.779174 18114 consensus_queue.cc:1035] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4967a29a76ba4b91a1ce8cf6d3bed7bd" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 43215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:12:04.786449 17690 tablet.cc:2369] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:04.796324 18118 mvcc.cc:205] Tried to move back new op lower bound from 7315368856676585472 to 7315368856487591936. Current Snapshot: MvccSnapshot[applied={T|T < 7315368856676585472}]
W20260806 01:12:04.797983 17883 tablet.cc:2369] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:04.799450 18121 mvcc.cc:205] Tried to move back new op lower bound from 7315368856676585472 to 7315368856487591936. Current Snapshot: MvccSnapshot[applied={T|T < 7315368856676585472}]
I20260806 01:12:04.816244 17419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:04.816497 17419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:04.820222 17419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ec5287d13504203a84822b7481b6558: Sending DeleteTablet for 3 replicas of tablet 49f9adad798f4336b953a1f27bb9a78b
I20260806 01:12:04.821681 17973 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f9adad798f4336b953a1f27bb9a78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:04 UTC) from {username='slave'} at 127.0.0.1:55170
I20260806 01:12:04.821749 17790 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f9adad798f4336b953a1f27bb9a78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:04 UTC) from {username='slave'} at 127.0.0.1:58280
I20260806 01:12:04.821983 17601 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f9adad798f4336b953a1f27bb9a78b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:04 UTC) from {username='slave'} at 127.0.0.1:60018
I20260806 01:12:04.824405 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 17507
I20260806 01:12:04.836380 18133 tablet_replica.cc:330] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: stopping tablet replica
I20260806 01:12:04.836961 18133 raft_consensus.cc:2229] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:04.837164 18131 tablet_replica.cc:330] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: stopping tablet replica
I20260806 01:12:04.837558 18133 raft_consensus.cc:2258] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:04.837539 18131 raft_consensus.cc:2229] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:04.838276 18131 raft_consensus.cc:2258] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:04.839857 18132 tablet_replica.cc:330] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: stopping tablet replica
I20260806 01:12:04.839851 18133 ts_tablet_manager.cc:1905] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:04.840236 18132 raft_consensus.cc:2229] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:04.840728 18132 raft_consensus.cc:2258] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:04.842859 18132 ts_tablet_manager.cc:1905] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:04.845916 18131 ts_tablet_manager.cc:1905] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:04.847304 18133 ts_tablet_manager.cc:1918] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:04.847486 18133 log.cc:1196] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/49f9adad798f4336b953a1f27bb9a78b
I20260806 01:12:04.848054 18133 ts_tablet_manager.cc:1939] T 49f9adad798f4336b953a1f27bb9a78b P 44b51cd5df8a44a2ab86e39e48f435bb: Deleting consensus metadata
I20260806 01:12:04.849951 18132 ts_tablet_manager.cc:1918] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:04.850116 18132 log.cc:1196] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/49f9adad798f4336b953a1f27bb9a78b
I20260806 01:12:04.850406 17393 catalog_manager.cc:4841] TS 44b51cd5df8a44a2ab86e39e48f435bb (127.22.63.65:37449): tablet 49f9adad798f4336b953a1f27bb9a78b (table test-table [id=e63b176bf54a40e4b776a64678274fa2]) successfully deleted
I20260806 01:12:04.850704 18132 ts_tablet_manager.cc:1939] T 49f9adad798f4336b953a1f27bb9a78b P 4967a29a76ba4b91a1ce8cf6d3bed7bd: Deleting consensus metadata
I20260806 01:12:04.852504 17392 catalog_manager.cc:4841] TS 4967a29a76ba4b91a1ce8cf6d3bed7bd (127.22.63.66:43215): tablet 49f9adad798f4336b953a1f27bb9a78b (table test-table [id=e63b176bf54a40e4b776a64678274fa2]) successfully deleted
I20260806 01:12:04.853068 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.826316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.22.63.65:37449)
0806 01:12:04.826521 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:04.826527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:04.839150 (+ 12623us) server_negotiation.cc:407] Connection header received
0806 01:12:04.839402 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:04.839412 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:04.839538 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:04.839663 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:04.840213 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.841593 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.846183 (+ 4590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:04.847518 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:04.847644 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:04.850742 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:04.850753 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:04.850785 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:04.851631 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:04.851700 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:04.851761 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:04.851899 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:04.852383 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 01:12:04.852524 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 01:12:04.853873 17682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:04.858413 18131 ts_tablet_manager.cc:1918] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:04.858620 18131 log.cc:1196] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/49f9adad798f4336b953a1f27bb9a78b
I20260806 01:12:04.859125 18131 ts_tablet_manager.cc:1939] T 49f9adad798f4336b953a1f27bb9a78b P b02c3ad0551946beb01aad3f2d4a5931: Deleting consensus metadata
I20260806 01:12:04.860674 17392 catalog_manager.cc:4841] TS b02c3ad0551946beb01aad3f2d4a5931 (127.22.63.67:44769): tablet 49f9adad798f4336b953a1f27bb9a78b (table test-table [id=e63b176bf54a40e4b776a64678274fa2]) successfully deleted
I20260806 01:12:04.968134 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 17507
I20260806 01:12:04.998920 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 17694
I20260806 01:12:05.006014 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:04.999845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.22.63.66:43215)
0806 01:12:05.000044 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:05.000050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:05.000073 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:12:05.000150 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:05.000156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:05.000260 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:05.000377 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:05.000810 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:05.001587 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:05.002864 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:05.003282 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:05.003415 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:05.004606 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:05.004614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:05.004628 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:05.005233 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:05.005278 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:05.005298 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:05.005435 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:05.005574 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 01:12:05.005678 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:12:05.006800 17876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:05.097134 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 17694
I20260806 01:12:05.129665 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 17886
I20260806 01:12:05.138041 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:05.130557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.22.63.67:44769)
0806 01:12:05.130778 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:05.130782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:05.130801 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:12:05.131062 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:05.131067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:05.131162 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:05.131257 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:05.131813 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:05.132621 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:05.134576 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:05.135028 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:05.135145 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:05.136536 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:05.136545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:05.136561 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:05.137201 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:05.137250 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:05.137283 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:05.137381 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:05.137564 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 01:12:05.137652 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:12:05.138937 18057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:05.261243 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 17886
I20260806 01:12:05.290947 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 17369
I20260806 01:12:05.292240 17465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:05.403997 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 17369
2026-08-06T01:12:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3977 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:05 dist-test-slave-7f01 krb5kdc[18145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:05 dist-test-slave-7f01 krb5kdc[18145](info): set up 2 sockets
Aug 06 01:12:05 dist-test-slave-7f01 krb5kdc[18145](info): commencing operation
krb5kdc: starting...
W20260806 01:12:06.736541 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:06 dist-test-slave-7f01 krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:07.074534 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40549
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:07.430996 18166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:07.431341 18166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:07.431525 18166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:07.439030 18166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:07.439146 18166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:07.439253 18166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:07.439364 18166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:07.439450 18166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:07.449821 18166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:07.451277 18166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:07.452705 18166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:07.465516 18173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:07.465543 18175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:07.465605 18172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:07.466578 18166 server_base.cc:1034] running on GCE node
I20260806 01:12:07.467489 18166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:07.469027 18166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:07.470275 18166 hybrid_clock.cc:648] HybridClock initialized: now 1785978727470274 us; error 113 us; skew 500 ppm
I20260806 01:12:07.470681 18166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:07 dist-test-slave-7f01 krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978727, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:07.473970 18166 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:07.475664 18166 webserver.cc:460] Webserver started at http://127.22.63.126:41553/ using document root <none> and password file <none>
I20260806 01:12:07.476432 18166 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:07.476593 18166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:07.476926 18166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:07.480217 18166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "a719ac5218e645859a32c6e91754011e"
format_stamp: "Formatted at 2026-08-06 01:12:07 on dist-test-slave-7f01"
server_key: "8bc6b61f735d675004c32e7fc65ce271"
server_key_iv: "79a31fc161bd7fa9f1277fd05eb3840d"
server_key_version: "encryptionkey@0"
I20260806 01:12:07.481004 18166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "a719ac5218e645859a32c6e91754011e"
format_stamp: "Formatted at 2026-08-06 01:12:07 on dist-test-slave-7f01"
server_key: "8bc6b61f735d675004c32e7fc65ce271"
server_key_iv: "79a31fc161bd7fa9f1277fd05eb3840d"
server_key_version: "encryptionkey@0"
I20260806 01:12:07.486304 18166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:12:07.489913 18184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:07.491277 18166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:07.491466 18166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a719ac5218e645859a32c6e91754011e"
format_stamp: "Formatted at 2026-08-06 01:12:07 on dist-test-slave-7f01"
server_key: "8bc6b61f735d675004c32e7fc65ce271"
server_key_iv: "79a31fc161bd7fa9f1277fd05eb3840d"
server_key_version: "encryptionkey@0"
I20260806 01:12:07.491677 18166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:07.510378 18166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:07.514760 18166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:07.515079 18166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:07.534871 18166 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40549
I20260806 01:12:07.534915 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40549 every 8 connection(s)
I20260806 01:12:07.536883 18166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:07.538789 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 18166
I20260806 01:12:07.538978 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:07.539602 22781 external_mini_cluster.cc:1428] Setting key a1ec9c3559774d7a2ee90455ec76c85b
I20260806 01:12:07.542063 18267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:07.556762 18267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: Bootstrap starting.
Aug 06 01:12:07 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:07.560371 18267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:07.561693 18267 log.cc:824] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:07.565104 18267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: No bootstrap required, opened a new log
I20260806 01:12:07.569473 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:07.542444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.22.63.126:40549)
0806 01:12:07.547451 (+ 5007us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:07.547470 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:07.547537 (+ 67us) server_negotiation.cc:407] Connection header received
0806 01:12:07.550263 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:07.550294 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:07.550753 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:07.551199 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:07.552891 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:07.554234 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:07.555703 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:07.556444 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:07.556667 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:07.561383 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:07.561390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:07.561409 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:07.561453 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:07.564959 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:07.565889 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:07.565895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:07.565901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:07.566079 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:07.566540 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:07.566544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:07.566546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:07.567140 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:07.567374 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:07.567995 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 01:12:07.568424 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4711,"thread_start_us":4532,"threads_started":1}
I20260806 01:12:07.573711 18267 raft_consensus.cc:348] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } }
I20260806 01:12:07.574100 18267 raft_consensus.cc:374] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:07.574218 18267 raft_consensus.cc:729] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a719ac5218e645859a32c6e91754011e, State: Initialized, Role: FOLLOWER
I20260806 01:12:07.574765 18267 consensus_queue.cc:260] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } }
I20260806 01:12:07.575009 18267 raft_consensus.cc:388] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:07.575143 18267 raft_consensus.cc:482] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:07.575335 18267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:07.577621 18267 raft_consensus.cc:504] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } }
I20260806 01:12:07.578123 18267 leader_election.cc:302] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a719ac5218e645859a32c6e91754011e; no voters:
I20260806 01:12:07.578711 18267 leader_election.cc:288] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:07.578853 18274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:07.579576 18274 raft_consensus.cc:686] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [term 1 LEADER]: Becoming Leader. State: Replica: a719ac5218e645859a32c6e91754011e, State: Running, Role: LEADER
I20260806 01:12:07.580164 18274 consensus_queue.cc:237] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } }
I20260806 01:12:07.580896 18267 sys_catalog.cc:564] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:07.583078 18280 sys_catalog.cc:455] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a719ac5218e645859a32c6e91754011e. Latest consensus state: current_term: 1 leader_uuid: "a719ac5218e645859a32c6e91754011e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } } }
I20260806 01:12:07.583575 18280 sys_catalog.cc:458] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:07.583452 18279 sys_catalog.cc:455] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a719ac5218e645859a32c6e91754011e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a719ac5218e645859a32c6e91754011e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40549 } } }
I20260806 01:12:07.584121 18279 sys_catalog.cc:458] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:07.585489 18286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:07.590976 18286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:07.600966 18286 catalog_manager.cc:1347] Generated new cluster ID: b22026a62d5941868b66de304cee3f3c
I20260806 01:12:07.601104 18286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:07.631515 18286 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:07.632999 18286 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:07.641979 18286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: Generated new TSK 0
I20260806 01:12:07.642850 18286 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:12:07.738838 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:08.119920 18307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:08.120210 18307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:08.120304 18307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:08.127568 18307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:08.127678 18307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:08.127856 18307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:12:08.137603 18307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:08.138906 18307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:08.140225 18307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:08.153537 18318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:08.153514 18320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:08.154271 18325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:08.154755 18307 server_base.cc:1034] running on GCE node
I20260806 01:12:08.155352 18307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:08.156327 18307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:08.157609 18307 hybrid_clock.cc:648] HybridClock initialized: now 1785978728157558 us; error 53 us; skew 500 ppm
I20260806 01:12:08.157923 18307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:08 dist-test-slave-7f01 krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:08.161046 18307 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:12:08.162564 18307 webserver.cc:460] Webserver started at http://127.22.63.65:33529/ using document root <none> and password file <none>
I20260806 01:12:08.163260 18307 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:08.163412 18307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:08.163865 18307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:08.166747 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "b552b091cfce407dba8fddec3ed63fec"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "b1d0ea9f6c4037e1c18713d6a026e2fc"
server_key_iv: "230a6adc1ffec8d92419609fd9f8eb36"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.167558 18307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "b552b091cfce407dba8fddec3ed63fec"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "b1d0ea9f6c4037e1c18713d6a026e2fc"
server_key_iv: "230a6adc1ffec8d92419609fd9f8eb36"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.172734 18307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:12:08.176219 18334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.177544 18307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:12:08.177728 18307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "b552b091cfce407dba8fddec3ed63fec"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "b1d0ea9f6c4037e1c18713d6a026e2fc"
server_key_iv: "230a6adc1ffec8d92419609fd9f8eb36"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.177933 18307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:08.205920 18307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:08.210105 18307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:08.210413 18307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:08.211410 18307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:08.212981 18307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:08.213090 18307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.213244 18307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:08.213330 18307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.248198 18307 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:40441
I20260806 01:12:08.248258 18500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:40441 every 8 connection(s)
I20260806 01:12:08.249836 18307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:12:08.259769 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 18307
I20260806 01:12:08.259986 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:12:08.260576 22781 external_mini_cluster.cc:1428] Setting key 9bfac0b5466a1dcbebad39fc8a0cc8d6
Aug 06 01:12:08 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:08.272586 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:08.253029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:39733 (local address 127.22.63.126:40549)
0806 01:12:08.253500 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:08.253505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:08.254505 (+ 1000us) server_negotiation.cc:407] Connection header received
0806 01:12:08.256457 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:08.256463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:08.256558 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:08.256685 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:08.258191 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.258935 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.261251 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.261747 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.261856 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:08.265500 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:08.265509 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:08.265516 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:08.265560 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:08.269168 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:08.269908 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:08.269915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:08.269921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:08.270096 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:08.270531 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:08.270538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:08.270544 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:08.270937 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:08.271108 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:08.271824 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 01:12:08.272112 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":228,"threads_started":1}
I20260806 01:12:08.273254 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:08.253628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40549 (local address 127.22.63.65:39733)
0806 01:12:08.254340 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:08.254377 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:08.256178 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:08.256930 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:08.256938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:08.257451 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:08.258017 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:08.258034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.259093 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.259098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:08.261081 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:08.261097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.261998 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.262006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:08.262284 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:08.263025 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:08.263048 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:08.265279 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:08.269327 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:08.269334 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:08.269348 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:08.269774 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:08.270214 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:08.270217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:08.270220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:08.270399 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:08.271071 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:08.271076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:08.271387 (+ 311us) client_negotiation.cc:769] Sending connection context
0806 01:12:08.271627 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 01:12:08.272099 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":290,"threads_started":1}
I20260806 01:12:08.274864 18502 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40549
I20260806 01:12:08.275226 18502 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:08.276145 18502 heartbeater.cc:511] Master 127.22.63.126:40549 requested a full tablet report, sending...
I20260806 01:12:08.278795 18217 ts_manager.cc:194] Registered new tserver with Master: b552b091cfce407dba8fddec3ed63fec (127.22.63.65:40441)
I20260806 01:12:08.281078 18217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:39733
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:12:08.334753 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:08.690975 18515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:08.691277 18515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:08.691435 18515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:08.698939 18515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:08.699050 18515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:08.699242 18515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:12:08.709187 18515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:08.710533 18515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:08.711946 18515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:08.726265 18524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:08.726310 18522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:08.726359 18520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:08.727304 18515 server_base.cc:1034] running on GCE node
I20260806 01:12:08.727981 18515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:08.728940 18515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:08.730158 18515 hybrid_clock.cc:648] HybridClock initialized: now 1785978728730098 us; error 50 us; skew 500 ppm
I20260806 01:12:08.730528 18515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:08 dist-test-slave-7f01 krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:08.734210 18515 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:12:08.736061 18515 webserver.cc:460] Webserver started at http://127.22.63.66:46337/ using document root <none> and password file <none>
I20260806 01:12:08.736836 18515 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:08.736990 18515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:08.737365 18515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:08.740773 18515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "378d0e3b098d44c9b766313c2911d46b"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "4ac89322e8b06ef8e02d5bdcdf632fed"
server_key_iv: "a729707eed50ae96a733ed7d67835753"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.741746 18515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "378d0e3b098d44c9b766313c2911d46b"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "4ac89322e8b06ef8e02d5bdcdf632fed"
server_key_iv: "a729707eed50ae96a733ed7d67835753"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.747534 18515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260806 01:12:08.752092 18532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.753619 18515 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 01:12:08.753821 18515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "378d0e3b098d44c9b766313c2911d46b"
format_stamp: "Formatted at 2026-08-06 01:12:08 on dist-test-slave-7f01"
server_key: "4ac89322e8b06ef8e02d5bdcdf632fed"
server_key_iv: "a729707eed50ae96a733ed7d67835753"
server_key_version: "encryptionkey@0"
I20260806 01:12:08.754055 18515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:08.778002 18515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:08.782967 18515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:08.783288 18515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:08.784427 18515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:08.785876 18515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:08.786011 18515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.786175 18515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:08.786228 18515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:08.822314 18515 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:42557
I20260806 01:12:08.822410 18711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:42557 every 8 connection(s)
I20260806 01:12:08.823661 18515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:08.827190 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 18515
I20260806 01:12:08.827406 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:08.827921 22781 external_mini_cluster.cc:1428] Setting key 60e2b908c29a44d2ca0771f6f54905c7
Aug 06 01:12:08 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:08.848168 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:08.826924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40549 (local address 127.22.63.66:59055)
0806 01:12:08.827803 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:08.827849 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:08.829834 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:08.830545 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:08.830552 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:08.831075 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:08.831657 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:08.831674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.833199 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.833203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:08.834466 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:08.834479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.835567 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.835571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:08.835833 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:08.836553 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:08.836576 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:08.839151 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:08.843473 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:08.843486 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:08.843506 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:08.844188 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:08.844639 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:08.844646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:08.844651 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:08.844909 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:08.845698 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:08.845707 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:08.846230 (+ 523us) client_negotiation.cc:769] Sending connection context
0806 01:12:08.846585 (+ 355us) client_negotiation.cc:240] Negotiation successful
0806 01:12:08.847037 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":359,"threads_started":1}
I20260806 01:12:08.848232 18717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:08.826356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:59055 (local address 127.22.63.126:40549)
0806 01:12:08.826869 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:08.826878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:08.828063 (+ 1185us) server_negotiation.cc:407] Connection header received
0806 01:12:08.830132 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:08.830138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:08.830219 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:08.830337 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:08.831861 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.833031 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.834620 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:08.835357 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:08.835498 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:08.839413 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:08.839422 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:08.839430 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:08.839479 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:08.843300 (+ 3821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:08.844335 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:08.844344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:08.844349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:08.844517 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:08.845080 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:08.845088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:08.845093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:08.845517 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:08.845689 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:08.846761 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0806 01:12:08.847691 (+ 930us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":209,"threads_started":1}
I20260806 01:12:08.849772 18712 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40549
I20260806 01:12:08.850118 18712 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:08.850854 18712 heartbeater.cc:511] Master 127.22.63.126:40549 requested a full tablet report, sending...
I20260806 01:12:08.852373 18217 ts_manager.cc:194] Registered new tserver with Master: 378d0e3b098d44c9b766313c2911d46b (127.22.63.66:42557)
I20260806 01:12:08.853226 18217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:59055
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:12:08.903167 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:09.252453 18722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:09.252876 18722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:09.253041 18722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:09.261787 18722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:09.261969 18722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:09.262148 18722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:09.272286 18722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:09.273669 18722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:09.275069 18722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 01:12:09.284370 18502 heartbeater.cc:503] Master 127.22.63.126:40549 was elected leader, sending a full tablet report...
W20260806 01:12:09.289165 18729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:09.289091 18732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:09.289130 18730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:09.290233 18722 server_base.cc:1034] running on GCE node
I20260806 01:12:09.290918 18722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:09.291941 18722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:09.293181 18722 hybrid_clock.cc:648] HybridClock initialized: now 1785978729293136 us; error 36 us; skew 500 ppm
I20260806 01:12:09.293562 18722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:09 dist-test-slave-7f01 krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978729, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:09.300549 18722 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:12:09.302155 18722 webserver.cc:460] Webserver started at http://127.22.63.67:39833/ using document root <none> and password file <none>
I20260806 01:12:09.302897 18722 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:09.303038 18722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:09.303375 18722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:09.306409 18722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "b439a0d657184ab9b1f8f1bb97e4a623"
format_stamp: "Formatted at 2026-08-06 01:12:09 on dist-test-slave-7f01"
server_key: "8e9fc961d68e2652ad76b812868c313e"
server_key_iv: "7e245b6983a6aa12d3d059ffc213ff97"
server_key_version: "encryptionkey@0"
I20260806 01:12:09.307231 18722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "b439a0d657184ab9b1f8f1bb97e4a623"
format_stamp: "Formatted at 2026-08-06 01:12:09 on dist-test-slave-7f01"
server_key: "8e9fc961d68e2652ad76b812868c313e"
server_key_iv: "7e245b6983a6aa12d3d059ffc213ff97"
server_key_version: "encryptionkey@0"
I20260806 01:12:09.312750 18722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:12:09.316603 18744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:09.318002 18722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:09.318185 18722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "b439a0d657184ab9b1f8f1bb97e4a623"
format_stamp: "Formatted at 2026-08-06 01:12:09 on dist-test-slave-7f01"
server_key: "8e9fc961d68e2652ad76b812868c313e"
server_key_iv: "7e245b6983a6aa12d3d059ffc213ff97"
server_key_version: "encryptionkey@0"
I20260806 01:12:09.318409 18722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:09.337920 18722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:09.342438 18722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:09.342753 18722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:09.343827 18722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:09.345351 18722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:09.345427 18722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:09.345589 18722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:09.345642 18722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:09.380483 18722 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:42123
I20260806 01:12:09.380575 18922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:42123 every 8 connection(s)
I20260806 01:12:09.381913 18722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:09.386027 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 18722
I20260806 01:12:09.386222 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:09.386818 22781 external_mini_cluster.cc:1428] Setting key a4b5e34bfca40c78875c9238aca61b14
Aug 06 01:12:09 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978729, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:09.401930 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.384552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:51169 (local address 127.22.63.126:40549)
0806 01:12:09.385086 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.385090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.385887 (+ 797us) server_negotiation.cc:407] Connection header received
0806 01:12:09.388057 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.388063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.388172 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.388292 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:09.389777 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.390547 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.392140 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.392557 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.392692 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.396065 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:09.396070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:09.396074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:09.396129 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:09.398772 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.399543 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.399549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.399553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.399681 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.400152 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.400157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.400160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.400437 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:09.400609 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.401293 (+ 684us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.401530 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":299,"threads_started":1}
I20260806 01:12:09.402827 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.385006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40549 (local address 127.22.63.67:51169)
0806 01:12:09.385705 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.385749 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.387787 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.388540 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.388548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.389058 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:09.389611 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.389627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.390704 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.390710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.391989 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.392002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.392806 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.392814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.393044 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.393665 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:09.393692 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:09.395857 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:09.398949 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:09.398959 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:09.398975 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:09.399409 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:09.399826 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:09.399831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:09.399835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:09.400030 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:09.400570 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:09.400578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:09.400922 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.401183 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.401709 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260806 01:12:09.404332 18923 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40549
I20260806 01:12:09.404634 18923 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:09.405352 18923 heartbeater.cc:511] Master 127.22.63.126:40549 requested a full tablet report, sending...
I20260806 01:12:09.406826 18217 ts_manager.cc:194] Registered new tserver with Master: b439a0d657184ab9b1f8f1bb97e4a623 (127.22.63.67:42123)
I20260806 01:12:09.407598 18217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:51169
I20260806 01:12:09.414286 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:09.432264 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.418644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.22.63.126:40549)
0806 01:12:09.418918 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.418924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.419316 (+ 392us) server_negotiation.cc:407] Connection header received
0806 01:12:09.419438 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.419442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.419520 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.419605 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:09.421246 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.422052 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.423472 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.423973 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.424078 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.426699 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:09.426704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:09.426707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:09.426738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:09.429500 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.430288 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.430292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.430294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.430412 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.430782 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.430785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.430787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.431079 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:09.431270 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.431679 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.431893 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:12:09.436257 18217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:09.443257 18217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:09.472527 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.455982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:42123 (local address 127.0.0.1:52520)
0806 01:12:09.456850 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.456891 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.457047 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.458410 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.458419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.458462 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:09.459026 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.459043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.465023 (+ 5980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.465032 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.469657 (+ 4625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.469676 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.471532 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.471538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.471769 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.471831 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.471960 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.472157 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":496,"threads_started":1}
I20260806 01:12:09.472564 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.456126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52520 (local address 127.22.63.67:42123)
0806 01:12:09.457610 (+ 1484us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.457619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.457656 (+ 37us) server_negotiation.cc:407] Connection header received
0806 01:12:09.457765 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.457773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.458031 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.460785 (+ 2754us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:09.461173 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.464808 (+ 3635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.469866 (+ 5058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.471330 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.471475 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.471720 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.472060 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.472227 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1301,"thread_start_us":1162,"threads_started":1}
I20260806 01:12:09.477494 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.457911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59480 (local address 127.22.63.65:40441)
0806 01:12:09.469729 (+ 11818us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.469742 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.469780 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:12:09.469890 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.469899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.470165 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.470369 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:09.471069 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.472583 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.475383 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.476375 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.476539 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.476771 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.476983 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.477137 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11589,"thread_start_us":11385,"threads_started":1}
I20260806 01:12:09.477523 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.457288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40441 (local address 127.0.0.1:59480)
0806 01:12:09.457803 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.457827 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.457997 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.470353 (+ 12356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.470358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.470393 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:09.470850 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.470861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.472806 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.472810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.475126 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.475142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.476594 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.476601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.476772 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.476812 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.476939 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.477068 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":1,"thread_start_us":339,"threads_started":1}
I20260806 01:12:09.476995 18817 tablet_service.cc:1467] Processing CreateTablet for tablet 271db5d912144a03b9adb73f81e387d3 (DEFAULT_TABLE table=test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:09.478600 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.457952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:42557 (local address 127.0.0.1:34770)
0806 01:12:09.468640 (+ 10688us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.468671 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.468818 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.470652 (+ 1834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.470658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.470694 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:09.471124 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.471133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.473200 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.473205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.475316 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.475330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.477411 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.477417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.477966 (+ 549us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.478000 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.478102 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.478277 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10509,"thread_start_us":10984,"threads_started":1}
I20260806 01:12:09.478811 18817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271db5d912144a03b9adb73f81e387d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:09.478950 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.458483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.22.63.66:42557)
0806 01:12:09.470040 (+ 11557us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.470048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.470075 (+ 27us) server_negotiation.cc:407] Connection header received
0806 01:12:09.470168 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.470175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.470396 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.470591 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:09.471283 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.472855 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.475480 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.476927 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.477072 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.477255 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.478160 (+ 905us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.478341 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11385,"thread_start_us":11246,"threads_started":1}
I20260806 01:12:09.481228 18404 tablet_service.cc:1467] Processing CreateTablet for tablet 271db5d912144a03b9adb73f81e387d3 (DEFAULT_TABLE table=test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:09.482717 18606 tablet_service.cc:1467] Processing CreateTablet for tablet 271db5d912144a03b9adb73f81e387d3 (DEFAULT_TABLE table=test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:09.483776 18404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271db5d912144a03b9adb73f81e387d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:09.484041 18606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271db5d912144a03b9adb73f81e387d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:09.494210 18948 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Bootstrap starting.
I20260806 01:12:09.497970 18948 tablet_bootstrap.cc:654] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:09.498895 18949 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Bootstrap starting.
I20260806 01:12:09.499331 18948 log.cc:824] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:09.500407 18950 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Bootstrap starting.
I20260806 01:12:09.503345 18949 tablet_bootstrap.cc:654] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:09.503473 18950 tablet_bootstrap.cc:654] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:09.504587 18949 log.cc:824] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:09.504739 18950 log.cc:824] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:09.505436 18948 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: No bootstrap required, opened a new log
I20260806 01:12:09.505714 18948 ts_tablet_manager.cc:1397] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260806 01:12:09.507922 18950 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: No bootstrap required, opened a new log
I20260806 01:12:09.507925 18949 tablet_bootstrap.cc:492] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: No bootstrap required, opened a new log
I20260806 01:12:09.508193 18950 ts_tablet_manager.cc:1397] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260806 01:12:09.508188 18949 ts_tablet_manager.cc:1397] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 01:12:09.514882 18950 raft_consensus.cc:348] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.515275 18950 raft_consensus.cc:374] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:09.515378 18950 raft_consensus.cc:729] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 378d0e3b098d44c9b766313c2911d46b, State: Initialized, Role: FOLLOWER
I20260806 01:12:09.516125 18950 consensus_queue.cc:260] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.516331 18948 raft_consensus.cc:348] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.516793 18948 raft_consensus.cc:374] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:09.516903 18948 raft_consensus.cc:729] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b552b091cfce407dba8fddec3ed63fec, State: Initialized, Role: FOLLOWER
I20260806 01:12:09.517481 18948 consensus_queue.cc:260] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.517680 18712 heartbeater.cc:503] Master 127.22.63.126:40549 was elected leader, sending a full tablet report...
I20260806 01:12:09.518489 18950 ts_tablet_manager.cc:1428] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260806 01:12:09.518983 18949 raft_consensus.cc:348] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.519418 18949 raft_consensus.cc:374] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:09.519518 18949 raft_consensus.cc:729] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b439a0d657184ab9b1f8f1bb97e4a623, State: Initialized, Role: FOLLOWER
I20260806 01:12:09.520105 18948 ts_tablet_manager.cc:1428] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260806 01:12:09.520234 18949 consensus_queue.cc:260] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.523393 18923 heartbeater.cc:503] Master 127.22.63.126:40549 was elected leader, sending a full tablet report...
I20260806 01:12:09.523986 18949 ts_tablet_manager.cc:1428] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260806 01:12:09.575826 18716 tablet.cc:2369] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:12:09.633806 18926 tablet.cc:2369] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:09.645749 18954 raft_consensus.cc:482] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:09.646085 18954 raft_consensus.cc:504] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.647878 18954 leader_election.cc:288] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b439a0d657184ab9b1f8f1bb97e4a623 (127.22.63.67:42123), b552b091cfce407dba8fddec3ed63fec (127.22.63.65:40441)
I20260806 01:12:09.656149 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.648471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:36979 (local address 127.22.63.67:42123)
0806 01:12:09.648701 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.648706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.649238 (+ 532us) server_negotiation.cc:407] Connection header received
0806 01:12:09.649611 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.649620 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.649722 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.649787 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:09.650663 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.651614 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.653792 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.655127 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.655277 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.655337 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.655825 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.655940 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:12:09.656246 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.648329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:42123 (local address 127.22.63.66:36979)
0806 01:12:09.649131 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.649154 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.649387 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.650029 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.650034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.650063 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:09.650475 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.650490 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.651784 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.651793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.653588 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.653601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.655377 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.655385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.655555 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.655592 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.655705 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.655874 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":478,"threads_started":1}
I20260806 01:12:09.657407 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.649665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:47135 (local address 127.22.63.65:40441)
0806 01:12:09.649880 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.649886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.650452 (+ 566us) server_negotiation.cc:407] Connection header received
0806 01:12:09.650797 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.650801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.650860 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.650972 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:09.651470 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.652620 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.655018 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.656281 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.656447 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.656527 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.656965 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.657104 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 01:12:09.657451 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.649272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40441 (local address 127.22.63.66:47135)
0806 01:12:09.650336 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:09.650364 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:09.650618 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:09.650989 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:09.650994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:09.651019 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:09.651314 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.651322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.652784 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.652790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.654826 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:09.654843 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.656620 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.656630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:09.656786 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.656813 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 01:12:09.656936 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 01:12:09.657076 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":651,"threads_started":1}
I20260806 01:12:09.657440 18845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271db5d912144a03b9adb73f81e387d3" candidate_uuid: "378d0e3b098d44c9b766313c2911d46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" is_pre_election: true
I20260806 01:12:09.658083 18845 raft_consensus.cc:2454] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378d0e3b098d44c9b766313c2911d46b in term 0.
I20260806 01:12:09.658495 18434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271db5d912144a03b9adb73f81e387d3" candidate_uuid: "378d0e3b098d44c9b766313c2911d46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b552b091cfce407dba8fddec3ed63fec" is_pre_election: true
I20260806 01:12:09.659042 18434 raft_consensus.cc:2454] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 378d0e3b098d44c9b766313c2911d46b in term 0.
I20260806 01:12:09.659025 18537 leader_election.cc:302] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378d0e3b098d44c9b766313c2911d46b, b439a0d657184ab9b1f8f1bb97e4a623; no voters:
I20260806 01:12:09.659463 18954 raft_consensus.cc:2781] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:09.659619 18954 raft_consensus.cc:482] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:09.659773 18954 raft_consensus.cc:3037] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:09.662055 18954 raft_consensus.cc:504] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.662686 18954 leader_election.cc:288] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [CANDIDATE]: Term 1 election: Requested vote from peers b439a0d657184ab9b1f8f1bb97e4a623 (127.22.63.67:42123), b552b091cfce407dba8fddec3ed63fec (127.22.63.65:40441)
I20260806 01:12:09.663182 18845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271db5d912144a03b9adb73f81e387d3" candidate_uuid: "378d0e3b098d44c9b766313c2911d46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b439a0d657184ab9b1f8f1bb97e4a623"
I20260806 01:12:09.663246 18434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271db5d912144a03b9adb73f81e387d3" candidate_uuid: "378d0e3b098d44c9b766313c2911d46b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b552b091cfce407dba8fddec3ed63fec"
I20260806 01:12:09.663417 18845 raft_consensus.cc:3037] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:09.663506 18434 raft_consensus.cc:3037] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:09.665704 18845 raft_consensus.cc:2454] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378d0e3b098d44c9b766313c2911d46b in term 1.
I20260806 01:12:09.665903 18434 raft_consensus.cc:2454] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 378d0e3b098d44c9b766313c2911d46b in term 1.
I20260806 01:12:09.666127 18537 leader_election.cc:302] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 378d0e3b098d44c9b766313c2911d46b, b439a0d657184ab9b1f8f1bb97e4a623; no voters:
I20260806 01:12:09.666440 18954 raft_consensus.cc:2781] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:09.666914 18954 raft_consensus.cc:686] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 1 LEADER]: Becoming Leader. State: Replica: 378d0e3b098d44c9b766313c2911d46b, State: Running, Role: LEADER
I20260806 01:12:09.667402 18954 consensus_queue.cc:237] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } }
I20260806 01:12:09.672413 18217 catalog_manager.cc:5495] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b reported cstate change: term changed from 0 to 1, leader changed from <none> to 378d0e3b098d44c9b766313c2911d46b (127.22.63.66). New cstate: current_term: 1 leader_uuid: "378d0e3b098d44c9b766313c2911d46b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "378d0e3b098d44c9b766313c2911d46b" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 42557 } health_report { overall_health: HEALTHY } } }
I20260806 01:12:09.709372 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.703477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.22.63.66:42557)
0806 01:12:09.703683 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.703688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.703707 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:12:09.703962 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.703966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.704086 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.704214 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:09.704736 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.705486 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.707095 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.707485 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.707582 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.708234 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:09.708789 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:09.708936 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.709099 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.709142 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:12:09.718106 18845 raft_consensus.cc:1261] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 1 FOLLOWER]: Refusing update from remote peer 378d0e3b098d44c9b766313c2911d46b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:09.718106 18434 raft_consensus.cc:1261] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 1 FOLLOWER]: Refusing update from remote peer 378d0e3b098d44c9b766313c2911d46b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:09.719153 18960 consensus_queue.cc:1035] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b439a0d657184ab9b1f8f1bb97e4a623" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 42123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:09.719597 18954 consensus_queue.cc:1035] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b552b091cfce407dba8fddec3ed63fec" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:09.730376 18964 mvcc.cc:205] Tried to move back new op lower bound from 7315368876913238016 to 7315368876721356800. Current Snapshot: MvccSnapshot[applied={T|T < 7315368876913238016}]
I20260806 01:12:09.734010 18965 mvcc.cc:205] Tried to move back new op lower bound from 7315368876913238016 to 7315368876721356800. Current Snapshot: MvccSnapshot[applied={T|T < 7315368876913238016}]
I20260806 01:12:09.735157 18967 mvcc.cc:205] Tried to move back new op lower bound from 7315368876913238016 to 7315368876721356800. Current Snapshot: MvccSnapshot[applied={T|T < 7315368876913238016}]
I20260806 01:12:09.746953 18217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:09.747278 18217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:09.751263 18217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a719ac5218e645859a32c6e91754011e: Sending DeleteTablet for 3 replicas of tablet 271db5d912144a03b9adb73f81e387d3
I20260806 01:12:09.752513 18817 tablet_service.cc:1514] Processing DeleteTablet for tablet 271db5d912144a03b9adb73f81e387d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:09 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:52520
I20260806 01:12:09.752671 18606 tablet_service.cc:1514] Processing DeleteTablet for tablet 271db5d912144a03b9adb73f81e387d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:09 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:34770
I20260806 01:12:09.752729 18404 tablet_service.cc:1514] Processing DeleteTablet for tablet 271db5d912144a03b9adb73f81e387d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:09 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:59480
I20260806 01:12:09.758280 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 18307
I20260806 01:12:09.768435 18977 tablet_replica.cc:330] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: stopping tablet replica
I20260806 01:12:09.769094 18977 raft_consensus.cc:2229] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:09.769780 18977 raft_consensus.cc:2258] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:09.772903 18977 ts_tablet_manager.cc:1905] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:09.782701 18977 ts_tablet_manager.cc:1918] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:09.782948 18977 log.cc:1196] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/271db5d912144a03b9adb73f81e387d3
I20260806 01:12:09.783655 18977 ts_tablet_manager.cc:1939] T 271db5d912144a03b9adb73f81e387d3 P b439a0d657184ab9b1f8f1bb97e4a623: Deleting consensus metadata
I20260806 01:12:09.786196 18191 catalog_manager.cc:4841] TS b439a0d657184ab9b1f8f1bb97e4a623 (127.22.63.67:42123): tablet 271db5d912144a03b9adb73f81e387d3 (table test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]) successfully deleted
I20260806 01:12:09.789175 18983 tablet_replica.cc:330] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: stopping tablet replica
I20260806 01:12:09.789434 18981 tablet_replica.cc:330] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: stopping tablet replica
I20260806 01:12:09.789686 18983 raft_consensus.cc:2229] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:09.789844 18981 raft_consensus.cc:2229] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:09.790262 18983 raft_consensus.cc:2258] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:09.790969 18981 raft_consensus.cc:2258] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:09.792650 18983 ts_tablet_manager.cc:1905] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:09.792675 18981 ts_tablet_manager.cc:1905] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 01:12:09 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:12:09.800316 18983 ts_tablet_manager.cc:1918] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:09.800509 18983 log.cc:1196] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/271db5d912144a03b9adb73f81e387d3
I20260806 01:12:09.800801 18981 ts_tablet_manager.cc:1918] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:09.800993 18981 log.cc:1196] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/271db5d912144a03b9adb73f81e387d3
I20260806 01:12:09.801163 18983 ts_tablet_manager.cc:1939] T 271db5d912144a03b9adb73f81e387d3 P b552b091cfce407dba8fddec3ed63fec: Deleting consensus metadata
I20260806 01:12:09.801662 18981 ts_tablet_manager.cc:1939] T 271db5d912144a03b9adb73f81e387d3 P 378d0e3b098d44c9b766313c2911d46b: Deleting consensus metadata
I20260806 01:12:09.803195 18191 catalog_manager.cc:4841] TS b552b091cfce407dba8fddec3ed63fec (127.22.63.65:40441): tablet 271db5d912144a03b9adb73f81e387d3 (table test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]) successfully deleted
I20260806 01:12:09.803534 18191 catalog_manager.cc:4841] TS 378d0e3b098d44c9b766313c2911d46b (127.22.63.66:42557): tablet 271db5d912144a03b9adb73f81e387d3 (table test-table [id=4226d4e705f54e1aba6dfad2fd217ad6]) successfully deleted
I20260806 01:12:09.804967 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.759442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.22.63.65:40441)
0806 01:12:09.759624 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.759630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.780645 (+ 21015us) server_negotiation.cc:407] Connection header received
0806 01:12:09.781224 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.781231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.781325 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.781406 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:09.784808 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.786058 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.788322 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.789112 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.789233 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.796582 (+ 7349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:09.796599 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:09.796618 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:09.796671 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:09.801188 (+ 4517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.802279 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.802288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.802292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.802482 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:09.802986 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:09.802993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:09.802997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:09.803708 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:09.804075 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:09.804301 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:12:09.804550 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:12:09.805651 18495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:09.956924 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 18307
I20260806 01:12:09.988260 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 18515
Aug 06 01:12:09 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:12:10.005354 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:09.989245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.22.63.66:42557)
0806 01:12:09.989421 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:09.989425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:09.989451 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:12:09.989575 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:09.989578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:09.989645 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:09.989727 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:09.991420 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.992353 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.993687 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:09.994142 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:09.994299 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:09.999165 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:09.999179 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:09.999194 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:09.999229 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:10.002407 (+ 3178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:10.003158 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:10.003165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:10.003168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:10.003329 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:10.003726 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:10.003731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:10.003760 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:10.004120 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:10.004525 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:10.004723 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 01:12:10.004957 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 01:12:10.006112 18707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:10.122542 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 18515
I20260806 01:12:10.150908 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 18722
Aug 06 01:12:10 dist-test-slave-7f01 krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:12:10.167457 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:10.151702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.22.63.67:42123)
0806 01:12:10.151953 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:10.151960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:10.152039 (+ 79us) server_negotiation.cc:407] Connection header received
0806 01:12:10.152294 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:10.152298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:10.152376 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:10.152497 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:10.154042 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:10.154774 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:10.156274 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:10.156667 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:10.156813 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:10.161361 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:10.161378 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:10.161395 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:10.161431 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:10.164567 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:10.165286 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:10.165295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:10.165297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:10.165443 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:10.165835 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:10.165840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:10.165843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:10.166192 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:10.166508 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:10.166857 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 01:12:10.167090 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 01:12:10.168186 18916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:10.257341 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 18722
I20260806 01:12:10.283241 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 18166
I20260806 01:12:10.284643 18261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:10.395835 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 18166
2026-08-06T01:12:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4995 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T01:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:10Z Disabled control of system clock
I20260806 01:12:10.497339 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:34367
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:34367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:10.847581 19001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:10.847954 19001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:10.848131 19001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:10.855263 19001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:10.855427 19001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:10.855499 19001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:10.855585 19001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:10.855679 19001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:10.865351 19001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:34367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:34367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:10.866828 19001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:10.868803 19001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:10.881364 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:10.881313 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:10.881402 19008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:10.882627 19001 server_base.cc:1034] running on GCE node
I20260806 01:12:10.883344 19001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:10.884824 19001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:10.886044 19001 hybrid_clock.cc:648] HybridClock initialized: now 1785978730886006 us; error 31 us; skew 500 ppm
I20260806 01:12:10.886364 19001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:10.888717 19001 webserver.cc:460] Webserver started at http://127.22.63.126:39723/ using document root <none> and password file <none>
I20260806 01:12:10.889372 19001 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:10.889472 19001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:10.889755 19001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:10.892745 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "842011e499964a03a6f788152af12a20"
format_stamp: "Formatted at 2026-08-06 01:12:10 on dist-test-slave-7f01"
server_key: "f35a2f44fad551bd174fd8151f271c83"
server_key_iv: "117dd2a862a4657b34d3a1b6db3ff5a4"
server_key_version: "encryptionkey@0"
I20260806 01:12:10.893523 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "842011e499964a03a6f788152af12a20"
format_stamp: "Formatted at 2026-08-06 01:12:10 on dist-test-slave-7f01"
server_key: "f35a2f44fad551bd174fd8151f271c83"
server_key_iv: "117dd2a862a4657b34d3a1b6db3ff5a4"
server_key_version: "encryptionkey@0"
I20260806 01:12:10.898783 19001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:12:10.902281 19029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:10.903679 19001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:10.903908 19001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "842011e499964a03a6f788152af12a20"
format_stamp: "Formatted at 2026-08-06 01:12:10 on dist-test-slave-7f01"
server_key: "f35a2f44fad551bd174fd8151f271c83"
server_key_iv: "117dd2a862a4657b34d3a1b6db3ff5a4"
server_key_version: "encryptionkey@0"
I20260806 01:12:10.904127 19001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:10.922259 19001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:10.923152 19001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:10.923481 19001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:10.943074 19001 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:34367
I20260806 01:12:10.943106 19099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:34367 every 8 connection(s)
I20260806 01:12:10.944617 19001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:10.948491 19101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:10.948755 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19001
I20260806 01:12:10.948945 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:10.949406 22781 external_mini_cluster.cc:1428] Setting key d970056ed0ff7b973d65f23f350d36a9
I20260806 01:12:10.962647 19101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: Bootstrap starting.
I20260806 01:12:10.964182 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:10.951292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37710 (local address 127.22.63.126:34367)
0806 01:12:10.952414 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:10.952425 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:10.952474 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:12:10.954720 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:10.954744 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:10.955217 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:10.955558 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:10.956270 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:10.957581 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:10.959590 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:10.960369 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:10.960516 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:10.961772 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:10.961779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:10.961792 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:10.962331 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:10.962406 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:10.962432 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:10.962520 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:10.962787 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 01:12:10.963170 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":445,"threads_started":1}
I20260806 01:12:10.966082 19101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:10.967119 19101 log.cc:824] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:10.969769 19101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: No bootstrap required, opened a new log
I20260806 01:12:10.976871 19101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } }
I20260806 01:12:10.977270 19101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:10.977401 19101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842011e499964a03a6f788152af12a20, State: Initialized, Role: FOLLOWER
I20260806 01:12:10.978045 19101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } }
I20260806 01:12:10.978315 19101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:10.978461 19101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:10.978641 19101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:10.980912 19101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } }
I20260806 01:12:10.981454 19101 leader_election.cc:302] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 842011e499964a03a6f788152af12a20; no voters:
I20260806 01:12:10.982046 19101 leader_election.cc:288] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:10.982261 19110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:10.983084 19110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [term 1 LEADER]: Becoming Leader. State: Replica: 842011e499964a03a6f788152af12a20, State: Running, Role: LEADER
I20260806 01:12:10.983596 19110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } }
I20260806 01:12:10.984265 19101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:10.986591 19111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "842011e499964a03a6f788152af12a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } } }
I20260806 01:12:10.987039 19111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:10.986923 19113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842011e499964a03a6f788152af12a20. Latest consensus state: current_term: 1 leader_uuid: "842011e499964a03a6f788152af12a20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842011e499964a03a6f788152af12a20" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 34367 } } }
I20260806 01:12:10.987210 19113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:10.988062 19120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:10.994423 19120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:11.003080 19120 catalog_manager.cc:1347] Generated new cluster ID: 956ece6a067b4c0980efc62e25f61717
I20260806 01:12:11.003293 19120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:11.043779 19120 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:11.045794 19120 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:11.055491 19120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: Generated new TSK 0
I20260806 01:12:11.056586 19120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:11.068039 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--builtin_ntp_servers=127.22.63.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:11.431442 19142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:11.431742 19142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:11.431910 19142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:11.439242 19142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:11.439388 19142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:11.439568 19142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:12:11.448918 19142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:11.450233 19142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:11.452116 19142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:11.471235 19148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:11.471243 19149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:11.479846 19154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:11.480283 19142 server_base.cc:1034] running on GCE node
I20260806 01:12:11.480933 19142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:11.484035 19142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:11.485287 19142 hybrid_clock.cc:648] HybridClock initialized: now 1785978731485232 us; error 49 us; skew 500 ppm
I20260806 01:12:11.485700 19142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:11.488160 19142 webserver.cc:460] Webserver started at http://127.22.63.65:37131/ using document root <none> and password file <none>
I20260806 01:12:11.488960 19142 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:11.489089 19142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:11.489418 19142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:11.492498 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "f01ef5828ddf4096ba44d0b35dcd216c"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "dcf014ce60d5a05275b7f5a92c4ccd40"
server_key_iv: "c8e0289151aa1e138ef2abd28041f265"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.493283 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "f01ef5828ddf4096ba44d0b35dcd216c"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "dcf014ce60d5a05275b7f5a92c4ccd40"
server_key_iv: "c8e0289151aa1e138ef2abd28041f265"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.498723 19142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:12:11.502395 19161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:11.503928 19142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:12:11.504094 19142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "f01ef5828ddf4096ba44d0b35dcd216c"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "dcf014ce60d5a05275b7f5a92c4ccd40"
server_key_iv: "c8e0289151aa1e138ef2abd28041f265"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.504309 19142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:11.518719 19142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:11.519506 19142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:11.519857 19142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:11.520869 19142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:11.522464 19142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:11.522554 19142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:11.522697 19142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:11.522797 19142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:11.559011 19142 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:46167
I20260806 01:12:11.559053 19330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:46167 every 8 connection(s)
I20260806 01:12:11.560405 19142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:12:11.566360 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19142
I20260806 01:12:11.566589 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:12:11.567265 22781 external_mini_cluster.cc:1428] Setting key f6da3ee44aff8a785f9ddf830666e76a
I20260806 01:12:11.570578 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--builtin_ntp_servers=127.22.63.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:11.576133 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:11.563400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:52847 (local address 127.22.63.126:34367)
0806 01:12:11.563977 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:11.563983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:11.565202 (+ 1219us) server_negotiation.cc:407] Connection header received
0806 01:12:11.567098 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:11.567103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:11.567196 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:11.567314 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:11.568578 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:11.569314 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:11.572336 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:11.573095 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:11.573239 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:11.575019 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:11.575028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:11.575036 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:11.575062 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:11.575116 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:11.575135 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:11.575225 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:11.575513 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 01:12:11.575629 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":296,"threads_started":1}
I20260806 01:12:11.577538 19337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:11.563976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34367 (local address 127.22.63.65:52847)
0806 01:12:11.565019 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:11.565073 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:11.566833 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:11.567566 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:11.567574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:11.567657 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:11.568413 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:11.568433 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:11.569511 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:11.569519 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:11.572158 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:11.572178 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:11.573349 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:11.573358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:11.573725 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:11.574603 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:11.574610 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:11.574670 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:11.574780 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:11.575242 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:11.575256 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:11.575267 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 01:12:11.575478 (+ 211us) client_negotiation.cc:240] Negotiation successful
0806 01:12:11.576005 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":343,"threads_started":1}
I20260806 01:12:11.579483 19331 heartbeater.cc:347] Connected to a master server at 127.22.63.126:34367
I20260806 01:12:11.579896 19331 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:11.581028 19331 heartbeater.cc:511] Master 127.22.63.126:34367 requested a full tablet report, sending...
I20260806 01:12:11.583772 19055 ts_manager.cc:194] Registered new tserver with Master: f01ef5828ddf4096ba44d0b35dcd216c (127.22.63.65:46167)
I20260806 01:12:11.585500 19055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.65:52847
W20260806 01:12:11.924965 19339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:11.925239 19339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:11.925329 19339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:11.932664 19339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:11.932781 19339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:11.932895 19339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:12:11.942022 19339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:11.943382 19339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:11.945345 19339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:11.959371 19345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:11.959393 19346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:11.959911 19352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:11.960165 19339 server_base.cc:1034] running on GCE node
I20260806 01:12:11.960798 19339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:11.961753 19339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:11.963002 19339 hybrid_clock.cc:648] HybridClock initialized: now 1785978731962958 us; error 57 us; skew 500 ppm
I20260806 01:12:11.963346 19339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:11.965809 19339 webserver.cc:460] Webserver started at http://127.22.63.66:39995/ using document root <none> and password file <none>
I20260806 01:12:11.966521 19339 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:11.966660 19339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:11.966992 19339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:11.969944 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "100d09429da743a5979ad30e67d4de6f"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "1485f68561450e7e8633a4c306f36692"
server_key_iv: "4591874d078fac430db960fda8c04279"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.970754 19339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "100d09429da743a5979ad30e67d4de6f"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "1485f68561450e7e8633a4c306f36692"
server_key_iv: "4591874d078fac430db960fda8c04279"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.976254 19339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:12:11.979724 19360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:11.981066 19339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:11.981249 19339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "100d09429da743a5979ad30e67d4de6f"
format_stamp: "Formatted at 2026-08-06 01:12:11 on dist-test-slave-7f01"
server_key: "1485f68561450e7e8633a4c306f36692"
server_key_iv: "4591874d078fac430db960fda8c04279"
server_key_version: "encryptionkey@0"
I20260806 01:12:11.981477 19339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:11.995071 19339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:11.996043 19339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:11.996356 19339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:11.997344 19339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:11.998914 19339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:11.999040 19339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:11.999176 19339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:11.999265 19339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:12.034804 19339 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:41167
I20260806 01:12:12.034857 19523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:41167 every 8 connection(s)
I20260806 01:12:12.036262 19339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:12.036929 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19339
I20260806 01:12:12.037144 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:12.037776 22781 external_mini_cluster.cc:1428] Setting key 3eafdcaf4b6f2454ac198ee92cd94cb8
I20260806 01:12:12.045121 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--builtin_ntp_servers=127.22.63.84:32847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:12.054600 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.041070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:37073 (local address 127.22.63.126:34367)
0806 01:12:12.041399 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.041408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.043363 (+ 1955us) server_negotiation.cc:407] Connection header received
0806 01:12:12.047351 (+ 3988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.047360 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.047489 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.047582 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:12.049212 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.049974 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.051690 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.052160 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.052298 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.053739 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:12.053745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:12.053748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:12.053768 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:12.053808 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:12.053820 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:12.053920 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.054161 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.054266 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 01:12:12.055729 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.041684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34367 (local address 127.22.63.66:37073)
0806 01:12:12.043138 (+ 1454us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.043213 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.046832 (+ 3619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.048207 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.048216 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.048316 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:12.049036 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.049057 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.050136 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.050142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.051539 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.051552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.052420 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.052431 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.052642 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.053399 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:12.053402 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:12.053442 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:12.053521 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:12.053922 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:12.053935 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:12.053945 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.054132 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.054534 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":5951,"threads_started":1}
I20260806 01:12:12.057250 19524 heartbeater.cc:347] Connected to a master server at 127.22.63.126:34367
I20260806 01:12:12.057570 19524 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:12.058305 19524 heartbeater.cc:511] Master 127.22.63.126:34367 requested a full tablet report, sending...
I20260806 01:12:12.059859 19055 ts_manager.cc:194] Registered new tserver with Master: 100d09429da743a5979ad30e67d4de6f (127.22.63.66:41167)
I20260806 01:12:12.060667 19055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.66:37073
W20260806 01:12:12.398604 19529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:12.398909 19529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:12.399062 19529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:12.406389 19529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:12.406494 19529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:12.406665 19529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:12.416118 19529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:32847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:34367
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:12.417518 19529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:12.419354 19529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:12.433020 19541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:12.432992 19542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:12.433084 19545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:12.433821 19529 server_base.cc:1034] running on GCE node
I20260806 01:12:12.434464 19529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:12.435420 19529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:12.436673 19529 hybrid_clock.cc:648] HybridClock initialized: now 1785978732436624 us; error 54 us; skew 500 ppm
I20260806 01:12:12.437019 19529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:12.439358 19529 webserver.cc:460] Webserver started at http://127.22.63.67:33653/ using document root <none> and password file <none>
I20260806 01:12:12.440155 19529 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:12.440285 19529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:12.440606 19529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:12.443449 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "42760ee67bb74e1aa02fb0590cfe53c3"
format_stamp: "Formatted at 2026-08-06 01:12:12 on dist-test-slave-7f01"
server_key: "62cf2a865ab8bc0bf68a5b6dd502e4fe"
server_key_iv: "71fb74bf5ca97f719848945b492af9d8"
server_key_version: "encryptionkey@0"
I20260806 01:12:12.444319 19529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "42760ee67bb74e1aa02fb0590cfe53c3"
format_stamp: "Formatted at 2026-08-06 01:12:12 on dist-test-slave-7f01"
server_key: "62cf2a865ab8bc0bf68a5b6dd502e4fe"
server_key_iv: "71fb74bf5ca97f719848945b492af9d8"
server_key_version: "encryptionkey@0"
I20260806 01:12:12.449373 19529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:12:12.452826 19552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:12.454099 19529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:12.454293 19529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "42760ee67bb74e1aa02fb0590cfe53c3"
format_stamp: "Formatted at 2026-08-06 01:12:12 on dist-test-slave-7f01"
server_key: "62cf2a865ab8bc0bf68a5b6dd502e4fe"
server_key_iv: "71fb74bf5ca97f719848945b492af9d8"
server_key_version: "encryptionkey@0"
I20260806 01:12:12.454530 19529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:12.481647 19529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:12.482520 19529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:12.482833 19529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:12.483911 19529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:12.485603 19529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:12.485723 19529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:12.485853 19529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:12.485939 19529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:12.520037 19529 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:41333
I20260806 01:12:12.520128 19720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:41333 every 8 connection(s)
I20260806 01:12:12.521389 19529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:12.521966 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19529
I20260806 01:12:12.522197 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:12.522842 22781 external_mini_cluster.cc:1428] Setting key 48e500ac70929621dca07147ff28ced4
I20260806 01:12:12.535351 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.524491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:52909 (local address 127.22.63.126:34367)
0806 01:12:12.524692 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.524702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.526323 (+ 1621us) server_negotiation.cc:407] Connection header received
0806 01:12:12.528437 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.528443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.528542 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.528664 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:12.529823 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.530650 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.532423 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.532878 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.533027 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.534417 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:12.534422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:12.534426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:12.534446 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:12.534479 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:12.534492 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:12.534594 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.534844 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.534967 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:12:12.536522 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.525134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:34367 (local address 127.22.63.67:52909)
0806 01:12:12.526153 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.526198 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.528155 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.528932 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.528940 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.529031 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:12.529638 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.529656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.530833 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.530837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.532272 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.532285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.533130 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.533139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.533389 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.534067 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:12.534070 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:12.534116 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:12.534196 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:12.534597 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:12.534613 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:12.534623 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.534838 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.535253 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":314,"threads_started":1}
I20260806 01:12:12.538121 19725 heartbeater.cc:347] Connected to a master server at 127.22.63.126:34367
I20260806 01:12:12.538475 19725 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:12.539283 19725 heartbeater.cc:511] Master 127.22.63.126:34367 requested a full tablet report, sending...
I20260806 01:12:12.540771 19055 ts_manager.cc:194] Registered new tserver with Master: 42760ee67bb74e1aa02fb0590cfe53c3 (127.22.63.67:41333)
I20260806 01:12:12.541530 19055 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.67:52909
I20260806 01:12:12.551100 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:12.570501 19334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.563682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.22.63.126:34367)
0806 01:12:12.563946 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.563952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.564324 (+ 372us) server_negotiation.cc:407] Connection header received
0806 01:12:12.564467 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.564476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.564571 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.564701 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:12.565218 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.566264 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.567665 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.568166 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.568270 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.569663 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:12.569671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:12.569677 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:12.569700 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:12.569759 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:12.569777 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:12.569946 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.570145 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.570206 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:12:12.574550 19055 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:12.581449 19055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:12.589658 19331 heartbeater.cc:503] Master 127.22.63.126:34367 was elected leader, sending a full tablet report...
I20260806 01:12:12.604501 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.593828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:41167 (local address 127.0.0.1:35440)
0806 01:12:12.595402 (+ 1574us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.595435 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.595628 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.596437 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.596443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.596468 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:12.596880 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.596898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.599111 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.599119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.601927 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.601940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.603322 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.603328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.603642 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.603686 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.603849 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.604066 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1374,"mutex_wait_us":68,"thread_start_us":1257,"threads_started":1}
I20260806 01:12:12.605319 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.593975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35440 (local address 127.22.63.66:41167)
0806 01:12:12.595025 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.595037 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.595546 (+ 509us) server_negotiation.cc:407] Connection header received
0806 01:12:12.595891 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.595904 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.596181 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.597135 (+ 954us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:12.597394 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.598917 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.602130 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.603110 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.603261 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.603527 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.604751 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.604945 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":744,"threads_started":1}
I20260806 01:12:12.607535 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.596630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.22.63.65:46167)
0806 01:12:12.597207 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.597218 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.600194 (+ 2976us) server_negotiation.cc:407] Connection header received
0806 01:12:12.600361 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.600372 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.600530 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.600701 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:12.601257 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.602621 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.605181 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.606277 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.606440 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.606701 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.606938 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.607115 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":441,"threads_started":1}
I20260806 01:12:12.607865 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.595783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:41333 (local address 127.0.0.1:44834)
0806 01:12:12.596352 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.596370 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.596467 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.599186 (+ 2719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.599190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.599217 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:12.599883 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.599899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.601858 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.601864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.604120 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.604135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.606886 (+ 2751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.606895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.607112 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.607165 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.607264 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.607486 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":1,"thread_start_us":280,"threads_started":1}
I20260806 01:12:12.607937 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.596187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.22.63.67:41333)
0806 01:12:12.598641 (+ 2454us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.598649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.598681 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:12:12.598770 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.598777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.598974 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.599193 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:12.600108 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.601705 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.604348 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.606681 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.606843 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.607126 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.607354 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.607521 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2284,"thread_start_us":2152,"threads_started":1}
I20260806 01:12:12.608804 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.596951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53270)
0806 01:12:12.600081 (+ 3130us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.600099 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.600224 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.600696 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.600700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.600726 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:12.601095 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.601104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.602803 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.602809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.605003 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.605018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.606487 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.606503 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.606696 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.606745 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.608331 (+ 1586us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.608460 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2977,"mutex_wait_us":3,"thread_start_us":2710,"threads_started":1}
I20260806 01:12:12.609402 19431 tablet_service.cc:1467] Processing CreateTablet for tablet fa7ae28c84264e7292a69085b6d31300 (DEFAULT_TABLE table=test-table [id=4e6564a0b17c4121b8d42cb6fa0f6f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:12.610877 19431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7ae28c84264e7292a69085b6d31300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:12.611625 19627 tablet_service.cc:1467] Processing CreateTablet for tablet fa7ae28c84264e7292a69085b6d31300 (DEFAULT_TABLE table=test-table [id=4e6564a0b17c4121b8d42cb6fa0f6f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:12.612855 19627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7ae28c84264e7292a69085b6d31300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:12.612777 19232 tablet_service.cc:1467] Processing CreateTablet for tablet fa7ae28c84264e7292a69085b6d31300 (DEFAULT_TABLE table=test-table [id=4e6564a0b17c4121b8d42cb6fa0f6f7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:12.614135 19232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa7ae28c84264e7292a69085b6d31300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:12.624157 19750 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Bootstrap starting.
I20260806 01:12:12.625849 19751 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Bootstrap starting.
I20260806 01:12:12.629029 19752 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Bootstrap starting.
I20260806 01:12:12.629570 19751 tablet_bootstrap.cc:654] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:12.629593 19750 tablet_bootstrap.cc:654] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:12.630887 19751 log.cc:824] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:12.630890 19750 log.cc:824] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:12.632123 19752 tablet_bootstrap.cc:654] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:12.633389 19752 log.cc:824] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:12.633946 19750 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: No bootstrap required, opened a new log
I20260806 01:12:12.634253 19750 ts_tablet_manager.cc:1397] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 01:12:12.634378 19751 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: No bootstrap required, opened a new log
I20260806 01:12:12.634657 19751 ts_tablet_manager.cc:1397] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 01:12:12.636958 19752 tablet_bootstrap.cc:492] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: No bootstrap required, opened a new log
I20260806 01:12:12.637226 19752 ts_tablet_manager.cc:1397] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260806 01:12:12.642103 19751 raft_consensus.cc:348] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.642493 19751 raft_consensus.cc:374] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:12.642601 19751 raft_consensus.cc:729] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100d09429da743a5979ad30e67d4de6f, State: Initialized, Role: FOLLOWER
I20260806 01:12:12.643215 19751 consensus_queue.cc:260] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.644390 19524 heartbeater.cc:503] Master 127.22.63.126:34367 was elected leader, sending a full tablet report...
I20260806 01:12:12.645123 19751 ts_tablet_manager.cc:1428] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
I20260806 01:12:12.645053 19750 raft_consensus.cc:348] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.645466 19750 raft_consensus.cc:374] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:12.645567 19750 raft_consensus.cc:729] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42760ee67bb74e1aa02fb0590cfe53c3, State: Initialized, Role: FOLLOWER
I20260806 01:12:12.646288 19750 consensus_queue.cc:260] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.648072 19752 raft_consensus.cc:348] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.648605 19752 raft_consensus.cc:374] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:12.648729 19752 raft_consensus.cc:729] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01ef5828ddf4096ba44d0b35dcd216c, State: Initialized, Role: FOLLOWER
I20260806 01:12:12.649473 19752 consensus_queue.cc:260] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.651648 19752 ts_tablet_manager.cc:1428] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 01:12:12.652740 19725 heartbeater.cc:503] Master 127.22.63.126:34367 was elected leader, sending a full tablet report...
I20260806 01:12:12.653208 19750 ts_tablet_manager.cc:1428] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Time spent starting tablet: real 0.019s user 0.009s sys 0.010s
I20260806 01:12:12.701794 19759 raft_consensus.cc:482] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:12.702168 19759 raft_consensus.cc:504] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.703948 19759 leader_election.cc:288] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42760ee67bb74e1aa02fb0590cfe53c3 (127.22.63.67:41333), f01ef5828ddf4096ba44d0b35dcd216c (127.22.63.65:46167)
I20260806 01:12:12.712945 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.704288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:60003 (local address 127.22.63.67:41333)
0806 01:12:12.704544 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.704553 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.705140 (+ 587us) server_negotiation.cc:407] Connection header received
0806 01:12:12.705375 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.705384 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.705498 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.705597 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:12.706277 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.707317 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.710544 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.711824 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.711989 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.712062 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.712578 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.712700 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:12:12.712945 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.705281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:54377 (local address 127.22.63.65:46167)
0806 01:12:12.705505 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.705513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.705878 (+ 365us) server_negotiation.cc:407] Connection header received
0806 01:12:12.706041 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.706044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.706116 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.706208 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:12.706698 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.707634 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.710544 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.711822 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.712005 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.712064 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.712557 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.712687 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:12:12.713111 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.705123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.22.63.66:54377)
0806 01:12:12.705761 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.705780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.705913 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.706229 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.706237 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.706268 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:12.706548 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.706555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.707778 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.707783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.710301 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.710317 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.712057 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.712064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.712298 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.712397 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.712497 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.712684 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":147,"thread_start_us":318,"threads_started":1}
I20260806 01:12:12.713124 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.704334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:41333 (local address 127.22.63.66:60003)
0806 01:12:12.704981 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.705019 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:12.705193 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:12.705659 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:12.705664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:12.705694 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:12.706112 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.706122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.707497 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.707502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.710301 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:12.710317 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.712032 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.712044 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:12.712303 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.712405 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 01:12:12.712497 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 01:12:12.712686 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":291,"thread_start_us":347,"threads_started":1}
I20260806 01:12:12.714346 19658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7ae28c84264e7292a69085b6d31300" candidate_uuid: "100d09429da743a5979ad30e67d4de6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" is_pre_election: true
I20260806 01:12:12.714334 19262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7ae28c84264e7292a69085b6d31300" candidate_uuid: "100d09429da743a5979ad30e67d4de6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" is_pre_election: true
I20260806 01:12:12.714968 19262 raft_consensus.cc:2454] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 100d09429da743a5979ad30e67d4de6f in term 0.
I20260806 01:12:12.714968 19658 raft_consensus.cc:2454] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 100d09429da743a5979ad30e67d4de6f in term 0.
I20260806 01:12:12.715677 19364 leader_election.cc:302] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 100d09429da743a5979ad30e67d4de6f, 42760ee67bb74e1aa02fb0590cfe53c3; no voters:
I20260806 01:12:12.716099 19759 raft_consensus.cc:2781] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:12.716279 19759 raft_consensus.cc:482] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:12.716399 19759 raft_consensus.cc:3037] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:12.718674 19759 raft_consensus.cc:504] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.719247 19759 leader_election.cc:288] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [CANDIDATE]: Term 1 election: Requested vote from peers 42760ee67bb74e1aa02fb0590cfe53c3 (127.22.63.67:41333), f01ef5828ddf4096ba44d0b35dcd216c (127.22.63.65:46167)
I20260806 01:12:12.719821 19658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7ae28c84264e7292a69085b6d31300" candidate_uuid: "100d09429da743a5979ad30e67d4de6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42760ee67bb74e1aa02fb0590cfe53c3"
I20260806 01:12:12.719898 19262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa7ae28c84264e7292a69085b6d31300" candidate_uuid: "100d09429da743a5979ad30e67d4de6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01ef5828ddf4096ba44d0b35dcd216c"
I20260806 01:12:12.720098 19262 raft_consensus.cc:3037] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:12.720082 19658 raft_consensus.cc:3037] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:12.723426 19658 raft_consensus.cc:2454] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 100d09429da743a5979ad30e67d4de6f in term 1.
I20260806 01:12:12.723450 19262 raft_consensus.cc:2454] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 100d09429da743a5979ad30e67d4de6f in term 1.
I20260806 01:12:12.723868 19364 leader_election.cc:302] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 100d09429da743a5979ad30e67d4de6f, 42760ee67bb74e1aa02fb0590cfe53c3; no voters:
I20260806 01:12:12.724128 19759 raft_consensus.cc:2781] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:12.724587 19759 raft_consensus.cc:686] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 1 LEADER]: Becoming Leader. State: Replica: 100d09429da743a5979ad30e67d4de6f, State: Running, Role: LEADER
I20260806 01:12:12.725065 19759 consensus_queue.cc:237] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } }
I20260806 01:12:12.729574 19055 catalog_manager.cc:5495] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 100d09429da743a5979ad30e67d4de6f (127.22.63.66). New cstate: current_term: 1 leader_uuid: "100d09429da743a5979ad30e67d4de6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "100d09429da743a5979ad30e67d4de6f" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 41167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 } health_report { overall_health: UNKNOWN } } }
I20260806 01:12:12.763376 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.757961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.22.63.66:41167)
0806 01:12:12.758223 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.758231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.758252 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:12:12.758344 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.758348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.758439 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.758561 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:12.758935 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.759801 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.761210 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.761623 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.761714 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.762265 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:12.762745 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:12.762893 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.763014 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.763076 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:12:12.771926 19262 raft_consensus.cc:1261] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 1 FOLLOWER]: Refusing update from remote peer 100d09429da743a5979ad30e67d4de6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:12.771942 19658 raft_consensus.cc:1261] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 1 FOLLOWER]: Refusing update from remote peer 100d09429da743a5979ad30e67d4de6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:12.772861 19767 consensus_queue.cc:1035] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "42760ee67bb74e1aa02fb0590cfe53c3" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 41333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:12.773211 19759 consensus_queue.cc:1035] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01ef5828ddf4096ba44d0b35dcd216c" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 01:12:12.773281 19731 tablet.cc:2369] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:12.798594 19055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:12.798902 19055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:12.802884 19055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 842011e499964a03a6f788152af12a20: Sending DeleteTablet for 3 replicas of tablet fa7ae28c84264e7292a69085b6d31300
I20260806 01:12:12.804121 19431 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7ae28c84264e7292a69085b6d31300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:12 UTC) from {username='slave'} at 127.0.0.1:35440
I20260806 01:12:12.804275 19627 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7ae28c84264e7292a69085b6d31300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:12 UTC) from {username='slave'} at 127.0.0.1:44834
I20260806 01:12:12.805156 19232 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7ae28c84264e7292a69085b6d31300 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:12 UTC) from {username='slave'} at 127.0.0.1:53270
I20260806 01:12:12.805899 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19142
I20260806 01:12:12.820534 19786 tablet_replica.cc:330] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: stopping tablet replica
I20260806 01:12:12.820777 19788 tablet_replica.cc:330] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: stopping tablet replica
I20260806 01:12:12.821039 19786 raft_consensus.cc:2229] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:12.821146 19788 raft_consensus.cc:2229] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:12.821631 19788 raft_consensus.cc:2258] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:12.821736 19786 raft_consensus.cc:2258] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:12.822453 19787 tablet_replica.cc:330] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: stopping tablet replica
I20260806 01:12:12.822868 19787 raft_consensus.cc:2229] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:12.823371 19787 raft_consensus.cc:2258] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:12.824636 19786 ts_tablet_manager.cc:1905] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:12.825525 19788 ts_tablet_manager.cc:1905] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:12.826035 19787 ts_tablet_manager.cc:1905] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:12.832290 19786 ts_tablet_manager.cc:1918] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:12.832484 19786 log.cc:1196] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/fa7ae28c84264e7292a69085b6d31300
I20260806 01:12:12.833060 19786 ts_tablet_manager.cc:1939] T fa7ae28c84264e7292a69085b6d31300 P 100d09429da743a5979ad30e67d4de6f: Deleting consensus metadata
I20260806 01:12:12.833737 19787 ts_tablet_manager.cc:1918] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:12.833959 19787 log.cc:1196] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/fa7ae28c84264e7292a69085b6d31300
I20260806 01:12:12.834084 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.807586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53284 (local address 127.22.63.65:46167)
0806 01:12:12.807830 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.807840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.813232 (+ 5392us) server_negotiation.cc:407] Connection header received
0806 01:12:12.814113 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.814120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.814228 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.814309 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:12.815510 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.816762 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.823800 (+ 7038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.824449 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.824583 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.832294 (+ 7711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:12.832307 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:12.832328 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:12.833100 (+ 772us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:12.833163 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:12.833197 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:12.833288 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.833585 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.833716 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:12:12.834774 19787 ts_tablet_manager.cc:1939] T fa7ae28c84264e7292a69085b6d31300 P 42760ee67bb74e1aa02fb0590cfe53c3: Deleting consensus metadata
I20260806 01:12:12.835187 19035 catalog_manager.cc:4841] TS 100d09429da743a5979ad30e67d4de6f (127.22.63.66:41167): tablet fa7ae28c84264e7292a69085b6d31300 (table test-table [id=4e6564a0b17c4121b8d42cb6fa0f6f7f]) successfully deleted
I20260806 01:12:12.835330 19321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:12.836707 19035 catalog_manager.cc:4841] TS 42760ee67bb74e1aa02fb0590cfe53c3 (127.22.63.67:41333): tablet fa7ae28c84264e7292a69085b6d31300 (table test-table [id=4e6564a0b17c4121b8d42cb6fa0f6f7f]) successfully deleted
I20260806 01:12:12.925243 19788 ts_tablet_manager.cc:1918] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:12.925554 19788 log.cc:1196] T fa7ae28c84264e7292a69085b6d31300 P f01ef5828ddf4096ba44d0b35dcd216c: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/fa7ae28c84264e7292a69085b6d31300
I20260806 01:12:12.926039 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19142
I20260806 01:12:12.957239 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19339
W20260806 01:12:12.957777 19035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:46167 (error 108)
I20260806 01:12:12.960565 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.960051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53286)
0806 01:12:12.960267 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:12.960402 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 01:12:12.960820 19035 catalog_manager.cc:4584] TS f01ef5828ddf4096ba44d0b35dcd216c (127.22.63.65:46167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa7ae28c84264e7292a69085b6d31300: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
I20260806 01:12:12.966617 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:12.958476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.22.63.66:41167)
0806 01:12:12.958690 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:12.958697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:12.958719 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:12:12.958787 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:12.958792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:12.958916 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:12.959033 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:12.959702 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.961070 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.962954 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:12.963635 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:12.963799 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:12.964957 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:12.964968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:12.964985 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:12.965780 (+ 795us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:12.965850 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:12.965876 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:12.965999 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:12.966138 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 01:12:12.966241 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:12:12.967438 19517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:13.014664 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.014074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53292)
0806 01:12:13.014330 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.014456 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 01:12:13.015853 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.015397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53308)
0806 01:12:13.015562 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.015645 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 01:12:13.076501 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.075883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53312)
0806 01:12:13.076149 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.076262 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 01:12:13.077651 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.077259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53318)
0806 01:12:13.077431 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.077515 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:12:13.081110 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19339
I20260806 01:12:13.108331 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19529
I20260806 01:12:13.115182 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.109227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.22.63.67:41333)
0806 01:12:13.109395 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:13.109400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:13.109438 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:12:13.109608 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:13.109612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:13.109706 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:13.109797 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:13.110200 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:13.110990 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:13.112328 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:13.112750 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:13.112882 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:13.113889 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:13.113898 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:13.113913 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:13.114516 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:13.114564 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:13.114588 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:13.114682 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:13.114833 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 01:12:13.114936 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 01:12:13.115991 19715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:13.158492 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.157962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53334)
0806 01:12:13.158186 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.158323 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 01:12:13.159588 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.159211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53342)
0806 01:12:13.159412 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.159478 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 01:12:13.206519 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19529
I20260806 01:12:13.231950 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19001
I20260806 01:12:13.233105 19095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:13.344058 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:13.343499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:46167 (local address 127.0.0.1:53344)
0806 01:12:13.343676 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:13.343835 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:46167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:12:13.344008 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19001
2026-08-06T01:12:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2949 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:13 dist-test-slave-7f01 krb5kdc[19804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:13 dist-test-slave-7f01 krb5kdc[19804](info): set up 2 sockets
Aug 06 01:12:13 dist-test-slave-7f01 krb5kdc[19804](info): commencing operation
krb5kdc: starting...
W20260806 01:12:14.694326 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:14 dist-test-slave-7f01 krb5kdc[19804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:15.049490 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40515
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:15.400975 19825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:15.401304 19825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:15.401480 19825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:15.408582 19825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:15.408684 19825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:15.408789 19825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:15.408891 19825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:15.408965 19825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:15.418996 19825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:15.420374 19825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:15.421691 19825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:15.434454 19832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:15.434370 19835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:15.434410 19837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:15.435618 19825 server_base.cc:1034] running on GCE node
I20260806 01:12:15.436486 19825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:15.437985 19825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:15.439332 19825 hybrid_clock.cc:648] HybridClock initialized: now 1785978735439256 us; error 54 us; skew 500 ppm
I20260806 01:12:15.439725 19825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:15 dist-test-slave-7f01 krb5kdc[19804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978735, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:15.443141 19825 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:15.444715 19825 webserver.cc:460] Webserver started at http://127.22.63.126:35239/ using document root <none> and password file <none>
I20260806 01:12:15.445391 19825 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:15.445497 19825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:15.445748 19825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:15.448828 19825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "bce9fd22453146abb816e4c4c0dfa87f"
format_stamp: "Formatted at 2026-08-06 01:12:15 on dist-test-slave-7f01"
server_key: "83250573e732eb3b79f13ef0e393974d"
server_key_iv: "e20907e71e8e905b80989986f8215272"
server_key_version: "encryptionkey@0"
I20260806 01:12:15.449550 19825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "bce9fd22453146abb816e4c4c0dfa87f"
format_stamp: "Formatted at 2026-08-06 01:12:15 on dist-test-slave-7f01"
server_key: "83250573e732eb3b79f13ef0e393974d"
server_key_iv: "e20907e71e8e905b80989986f8215272"
server_key_version: "encryptionkey@0"
I20260806 01:12:15.454612 19825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:12:15.458081 19847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:15.459329 19825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 01:12:15.459468 19825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "bce9fd22453146abb816e4c4c0dfa87f"
format_stamp: "Formatted at 2026-08-06 01:12:15 on dist-test-slave-7f01"
server_key: "83250573e732eb3b79f13ef0e393974d"
server_key_iv: "e20907e71e8e905b80989986f8215272"
server_key_version: "encryptionkey@0"
I20260806 01:12:15.459621 19825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:15.476613 19825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:15.480993 19825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:15.481242 19825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:15.500608 19825 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40515
I20260806 01:12:15.500653 19921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40515 every 8 connection(s)
I20260806 01:12:15.502058 19825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:15.503963 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19825
I20260806 01:12:15.504182 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:15.504779 22781 external_mini_cluster.cc:1428] Setting key a90f2f59cd18c11153db14dac9b9bd67
I20260806 01:12:15.506901 19924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:12:15 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:15.520339 19924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: Bootstrap starting.
I20260806 01:12:15.523896 19924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:15.524935 19924 log.cc:824] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:15.527871 19930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:15.506722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.22.63.126:40515)
0806 01:12:15.507555 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:15.507567 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:15.507609 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:12:15.509296 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:15.509317 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:15.509677 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:15.510003 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:15.511835 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:15.513182 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:15.514967 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:15.515605 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:15.515783 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:15.520681 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:15.520689 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:15.520702 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:15.520737 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:15.523673 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:15.524511 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:15.524518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:15.524524 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:15.524694 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:15.525117 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:15.525122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:15.525125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:15.525670 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:15.525997 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:15.526453 (+ 456us) server_negotiation.cc:299] Negotiation successful
0806 01:12:15.526874 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":406,"threads_started":1}
I20260806 01:12:15.527963 19924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: No bootstrap required, opened a new log
I20260806 01:12:15.535825 19924 raft_consensus.cc:348] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } }
I20260806 01:12:15.536253 19924 raft_consensus.cc:374] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:15.536375 19924 raft_consensus.cc:729] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce9fd22453146abb816e4c4c0dfa87f, State: Initialized, Role: FOLLOWER
I20260806 01:12:15.536849 19924 consensus_queue.cc:260] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } }
I20260806 01:12:15.537050 19924 raft_consensus.cc:388] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:15.537144 19924 raft_consensus.cc:482] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:15.537271 19924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:15.539317 19924 raft_consensus.cc:504] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } }
I20260806 01:12:15.539839 19924 leader_election.cc:302] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bce9fd22453146abb816e4c4c0dfa87f; no voters:
I20260806 01:12:15.540397 19924 leader_election.cc:288] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:15.540491 19932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:15.541276 19932 raft_consensus.cc:686] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [term 1 LEADER]: Becoming Leader. State: Replica: bce9fd22453146abb816e4c4c0dfa87f, State: Running, Role: LEADER
I20260806 01:12:15.541945 19932 consensus_queue.cc:237] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } }
I20260806 01:12:15.542822 19924 sys_catalog.cc:564] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:15.545130 19934 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bce9fd22453146abb816e4c4c0dfa87f. Latest consensus state: current_term: 1 leader_uuid: "bce9fd22453146abb816e4c4c0dfa87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } } }
I20260806 01:12:15.545229 19933 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bce9fd22453146abb816e4c4c0dfa87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce9fd22453146abb816e4c4c0dfa87f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40515 } } }
I20260806 01:12:15.545516 19934 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:15.545604 19933 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:15.547719 19942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:15.553035 19942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:15.562180 19942 catalog_manager.cc:1347] Generated new cluster ID: a2ee9ae320324e388fec795ddf4434e9
I20260806 01:12:15.562309 19942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:15.594799 19942 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:15.596293 19942 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:15.603298 19942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: Generated new TSK 0
I20260806 01:12:15.604372 19942 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.63.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
Entry for principal kudu/127.22.63.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab.
I20260806 01:12:15.683638 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:16.037521 19970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:16.037811 19970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:16.037964 19970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:16.045315 19970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:16.045472 19970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:16.045647 19970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:12:16.055423 19970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.65.keytab
--principal=kudu/127.22.63.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:16.056772 19970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:16.058066 19970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:16.071116 19985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:16.071170 19978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:16.071216 19975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:16.072065 19970 server_base.cc:1034] running on GCE node
I20260806 01:12:16.072679 19970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:16.073580 19970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:16.074798 19970 hybrid_clock.cc:648] HybridClock initialized: now 1785978736074761 us; error 48 us; skew 500 ppm
I20260806 01:12:16.075122 19970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:16 dist-test-slave-7f01 krb5kdc[19804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978736, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:16.078225 19970 init.cc:377] Logged in from keytab as kudu/127.22.63.65@KRBTEST.COM (short username kudu)
I20260806 01:12:16.079658 19970 webserver.cc:460] Webserver started at http://127.22.63.65:39117/ using document root <none> and password file <none>
I20260806 01:12:16.080364 19970 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:16.080511 19970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:16.080823 19970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:16.083659 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "e290ae07dd3f47b69097b102135b1fab"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "05f4e5480287ad4201d727a48c077d61"
server_key_iv: "74669bdeabf8874ab83e7b61762bd820"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.084482 19970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "e290ae07dd3f47b69097b102135b1fab"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "05f4e5480287ad4201d727a48c077d61"
server_key_iv: "74669bdeabf8874ab83e7b61762bd820"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.089571 19970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:12:16.092999 19992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.094307 19970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:16.094488 19970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "e290ae07dd3f47b69097b102135b1fab"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "05f4e5480287ad4201d727a48c077d61"
server_key_iv: "74669bdeabf8874ab83e7b61762bd820"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.094686 19970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:16.129097 19970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:16.133077 19970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:16.133368 19970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:16.134337 19970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:16.135658 19970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:16.135797 19970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.135950 19970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:16.136023 19970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.169279 19970 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:40259
I20260806 01:12:16.169329 20152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:40259 every 8 connection(s)
I20260806 01:12:16.170567 19970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:12:16.177290 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 19970
I20260806 01:12:16.177505 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:12:16.178117 22781 external_mini_cluster.cc:1428] Setting key 2fdecf6228ad87682bfd0d8ea62d574b
Aug 06 01:12:16 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978736, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.65@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:16.192148 20155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:16.173046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:47787 (local address 127.22.63.126:40515)
0806 01:12:16.173523 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:16.173527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:16.174321 (+ 794us) server_negotiation.cc:407] Connection header received
0806 01:12:16.176198 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:16.176204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:16.176292 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:16.176408 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:16.178377 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.179353 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.181457 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.181860 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.181964 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:16.186285 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:16.186290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:16.186294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:16.186327 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:16.189011 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:16.189757 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:16.189767 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:16.189771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:16.189895 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:16.190342 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:16.190351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:16.190354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:16.190623 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:16.190783 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:16.191511 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 01:12:16.191769 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260806 01:12:16.193086 20156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:16.173503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40515 (local address 127.22.63.65:47787)
0806 01:12:16.174166 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:16.174204 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:16.175929 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:16.176645 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:16.176653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:16.177348 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:16.178171 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:16.178200 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.179499 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.179503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:16.181283 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:16.181301 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.182110 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.182120 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:16.182457 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:16.183257 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:16.183299 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:16.186043 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:16.189166 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:16.189175 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:16.189192 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:16.189620 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:16.190016 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:16.190022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:16.190025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:16.190216 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:16.190755 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:16.190763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:16.191113 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 01:12:16.191376 (+ 263us) client_negotiation.cc:240] Negotiation successful
0806 01:12:16.191944 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":256,"threads_started":1}
I20260806 01:12:16.194670 20153 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40515
I20260806 01:12:16.195022 20153 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:16.195822 20153 heartbeater.cc:511] Master 127.22.63.126:40515 requested a full tablet report, sending...
I20260806 01:12:16.197898 19870 ts_manager.cc:194] Registered new tserver with Master: e290ae07dd3f47b69097b102135b1fab (127.22.63.65:40259)
I20260806 01:12:16.199550 19870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.65@KRBTEST.COM'} at 127.22.63.65:47787
WARNING: no policy specified for kudu/127.22.63.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
Entry for principal kudu/127.22.63.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab.
I20260806 01:12:16.247601 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:16.603284 20159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:16.603597 20159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:16.603798 20159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:16.610941 20159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:16.611094 20159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:16.611268 20159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:12:16.620757 20159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.66.keytab
--principal=kudu/127.22.63.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:16.621995 20159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:16.623322 20159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:16.636746 20168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:16.636788 20167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:16.636811 20170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:16.637585 20159 server_base.cc:1034] running on GCE node
I20260806 01:12:16.638159 20159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:16.639096 20159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:16.640304 20159 hybrid_clock.cc:648] HybridClock initialized: now 1785978736640250 us; error 36 us; skew 500 ppm
I20260806 01:12:16.640614 20159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:16 dist-test-slave-7f01 krb5kdc[19804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978736, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:16.643970 20159 init.cc:377] Logged in from keytab as kudu/127.22.63.66@KRBTEST.COM (short username kudu)
I20260806 01:12:16.645488 20159 webserver.cc:460] Webserver started at http://127.22.63.66:44273/ using document root <none> and password file <none>
I20260806 01:12:16.646184 20159 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:16.646291 20159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:16.646559 20159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:16.649726 20159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "b700f6f572c64396a957bd24797766f8"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "d17ba50e3d22473ebfb394ae4e515ba7"
server_key_iv: "92a584d2c0612dd77a52ae728a68c68e"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.650484 20159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "b700f6f572c64396a957bd24797766f8"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "d17ba50e3d22473ebfb394ae4e515ba7"
server_key_iv: "92a584d2c0612dd77a52ae728a68c68e"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.656013 20159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 01:12:16.659538 20179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.660959 20159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:16.661128 20159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "b700f6f572c64396a957bd24797766f8"
format_stamp: "Formatted at 2026-08-06 01:12:16 on dist-test-slave-7f01"
server_key: "d17ba50e3d22473ebfb394ae4e515ba7"
server_key_iv: "92a584d2c0612dd77a52ae728a68c68e"
server_key_version: "encryptionkey@0"
I20260806 01:12:16.661340 20159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:16.680922 20159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:16.685235 20159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:16.685559 20159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:16.686784 20159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:16.688292 20159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:16.688385 20159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.688530 20159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:16.688589 20159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:16.724138 20159 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:36921
I20260806 01:12:16.724231 20349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:36921 every 8 connection(s)
I20260806 01:12:16.725473 20159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:16.730535 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 20159
I20260806 01:12:16.730723 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:16.731191 22781 external_mini_cluster.cc:1428] Setting key fb518f2417086d149599be84647b718d
Aug 06 01:12:16 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978736, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.66@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:16.747025 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:16.728149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:57405 (local address 127.22.63.126:40515)
0806 01:12:16.728601 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:16.728605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:16.729488 (+ 883us) server_negotiation.cc:407] Connection header received
0806 01:12:16.731483 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:16.731488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:16.731577 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:16.731693 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:16.733521 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.734215 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.735773 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.736209 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.736348 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:16.740060 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:16.740071 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:16.740078 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:16.740124 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:16.743719 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:16.744457 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:16.744465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:16.744471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:16.744642 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:16.745042 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:16.745049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:16.745055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:16.745454 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:16.745609 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:16.746249 (+ 640us) server_negotiation.cc:299] Negotiation successful
0806 01:12:16.746510 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260806 01:12:16.747900 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:16.728634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40515 (local address 127.22.63.66:57405)
0806 01:12:16.729330 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:16.729368 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:16.731189 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:16.731965 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:16.731986 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:16.732687 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:16.733353 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:16.733369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.734374 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.734378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:16.735602 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:16.735614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:16.736448 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:16.736456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:16.736685 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:16.737358 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:16.737386 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:16.739818 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:16.743883 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:16.743890 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:16.743902 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:16.744344 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:16.744758 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:16.744761 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:16.744763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:16.744939 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:16.745581 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:16.745586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:16.745894 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 01:12:16.746118 (+ 224us) client_negotiation.cc:240] Negotiation successful
0806 01:12:16.746515 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":276,"threads_started":1}
I20260806 01:12:16.749317 20351 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40515
I20260806 01:12:16.749640 20351 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:16.750376 20351 heartbeater.cc:511] Master 127.22.63.126:40515 requested a full tablet report, sending...
I20260806 01:12:16.751945 19870 ts_manager.cc:194] Registered new tserver with Master: b700f6f572c64396a957bd24797766f8 (127.22.63.66:36921)
I20260806 01:12:16.753245 19870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.66@KRBTEST.COM'} at 127.22.63.66:57405
WARNING: no policy specified for kudu/127.22.63.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
Entry for principal kudu/127.22.63.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab.
I20260806 01:12:16.804931 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:17.157685 20359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:17.158018 20359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:17.158185 20359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:17.165766 20359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:17.165907 20359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:17.166065 20359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:17.175618 20359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.67.keytab
--principal=kudu/127.22.63.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:40515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:17.176795 20359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:17.178063 20359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:17.191447 20365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:17.191526 20366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:17.191491 20368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:17.192764 20359 server_base.cc:1034] running on GCE node
I20260806 01:12:17.193354 20359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:17.194309 20359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:17.195501 20359 hybrid_clock.cc:648] HybridClock initialized: now 1785978737195456 us; error 40 us; skew 500 ppm
I20260806 01:12:17.195883 20359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:17 dist-test-slave-7f01 krb5kdc[19804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978737, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:17.199175 20359 init.cc:377] Logged in from keytab as kudu/127.22.63.67@KRBTEST.COM (short username kudu)
I20260806 01:12:17.200856 20359 webserver.cc:460] Webserver started at http://127.22.63.67:34593/ using document root <none> and password file <none>
I20260806 01:12:17.201545 20359 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:17.201678 20359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:17.202005 20359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:17.202307 20153 heartbeater.cc:503] Master 127.22.63.126:40515 was elected leader, sending a full tablet report...
I20260806 01:12:17.205411 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa"
format_stamp: "Formatted at 2026-08-06 01:12:17 on dist-test-slave-7f01"
server_key: "8142e64d62cb04a85a2295e4e0526a56"
server_key_iv: "a3d2627477ec127657cd8a7d5b0ed58e"
server_key_version: "encryptionkey@0"
I20260806 01:12:17.206182 20359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa"
format_stamp: "Formatted at 2026-08-06 01:12:17 on dist-test-slave-7f01"
server_key: "8142e64d62cb04a85a2295e4e0526a56"
server_key_iv: "a3d2627477ec127657cd8a7d5b0ed58e"
server_key_version: "encryptionkey@0"
I20260806 01:12:17.211201 20359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:12:17.214577 20376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:17.215962 20359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 01:12:17.216130 20359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa"
format_stamp: "Formatted at 2026-08-06 01:12:17 on dist-test-slave-7f01"
server_key: "8142e64d62cb04a85a2295e4e0526a56"
server_key_iv: "a3d2627477ec127657cd8a7d5b0ed58e"
server_key_version: "encryptionkey@0"
I20260806 01:12:17.216324 20359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:17.239289 20359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:17.243861 20359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:17.244182 20359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:17.245185 20359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:17.246635 20359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:17.246738 20359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:17.246908 20359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:17.246992 20359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:17.281433 20359 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:43541
I20260806 01:12:17.281483 20544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:43541 every 8 connection(s)
I20260806 01:12:17.283215 20359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:17.287536 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 20359
I20260806 01:12:17.287771 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:17.288347 22781 external_mini_cluster.cc:1428] Setting key ab68cc6748e12e827008bfceca78407c
Aug 06 01:12:17 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978737, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.67@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:17.304145 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.286119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:45739 (local address 127.22.63.126:40515)
0806 01:12:17.286647 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.286651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.287550 (+ 899us) server_negotiation.cc:407] Connection header received
0806 01:12:17.290100 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.290105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.290204 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.290300 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:17.292092 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.292885 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.294381 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.294775 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.294873 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.298236 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:17.298241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:17.298245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:17.298295 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:17.300871 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.301568 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.301572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.301574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.301690 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.302216 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.302221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.302223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.302592 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:17.302722 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.303607 (+ 885us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.303855 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":285,"threads_started":1}
I20260806 01:12:17.305039 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.286529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:40515 (local address 127.22.63.67:45739)
0806 01:12:17.287355 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.287407 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.289771 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.290552 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.290561 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.291205 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:17.291937 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.291957 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.293019 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.293023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.294230 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.294242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.294966 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.294971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.295166 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.295837 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:12:17.295860 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:17.298037 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:17.301004 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:17.301011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:17.301022 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:17.301452 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:17.301813 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:12:17.301816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:12:17.301819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:12:17.302085 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:12:17.302737 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:17.302747 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:17.303189 (+ 442us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.303458 (+ 269us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.303954 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":296,"threads_started":1}
I20260806 01:12:17.306265 20545 heartbeater.cc:347] Connected to a master server at 127.22.63.126:40515
I20260806 01:12:17.306572 20545 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:17.307261 20545 heartbeater.cc:511] Master 127.22.63.126:40515 requested a full tablet report, sending...
I20260806 01:12:17.308663 19870 ts_manager.cc:194] Registered new tserver with Master: 2f9fc3c5822e4ca1ae75e08eae4c89fa (127.22.63.67:43541)
I20260806 01:12:17.309414 19870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.63.67@KRBTEST.COM'} at 127.22.63.67:45739
I20260806 01:12:17.315830 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:17.332118 20547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.319979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.22.63.126:40515)
0806 01:12:17.320146 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.320150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.320710 (+ 560us) server_negotiation.cc:407] Connection header received
0806 01:12:17.320869 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.320873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.320933 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.321028 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:17.322385 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.323082 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.324369 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.324742 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.324833 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.327256 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:17.327261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:17.327265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:17.327295 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:17.329692 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.330328 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.330332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.330334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.330445 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.330860 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.330863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.330865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.331180 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:17.331362 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.331659 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.331879 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:12:17.336005 19870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:17.342967 19870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:17.366961 20563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.356130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43541 (local address 127.0.0.1:60504)
0806 01:12:17.357755 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.357771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.357873 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.359281 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.359288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.359318 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.359758 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.359773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.361913 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.361923 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.364100 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.364116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.366008 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.366014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.366199 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.366236 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.366347 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.366484 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"mutex_wait_us":35,"thread_start_us":1106,"threads_started":1}
I20260806 01:12:17.367136 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.355443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.22.63.66:36921)
0806 01:12:17.356697 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.356708 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.357272 (+ 564us) server_negotiation.cc:407] Connection header received
0806 01:12:17.357719 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.357729 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.358011 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.358212 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.359186 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.360907 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.363870 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.365396 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.365553 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.365763 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.366597 (+ 834us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.366778 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":981,"threads_started":1}
I20260806 01:12:17.367769 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.355313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36921 (local address 127.0.0.1:46308)
0806 01:12:17.357119 (+ 1806us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.357156 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.357371 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.358500 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.358508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.358541 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.358972 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.358985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.361052 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.361060 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.363672 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.363688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.365583 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.365589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.365778 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.365835 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.365972 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.366173 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1622,"mutex_wait_us":41,"thread_start_us":2138,"threads_started":1}
I20260806 01:12:17.367805 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.356939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.22.63.67:43541)
0806 01:12:17.358687 (+ 1748us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.358699 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.358733 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:12:17.358829 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.358838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.359069 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.359976 (+ 907us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.360281 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.361796 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.364269 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.365723 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.365963 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.366191 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.366396 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.367399 (+ 1003us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1562,"thread_start_us":1168,"threads_started":1}
I20260806 01:12:17.368716 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.358664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40259 (local address 127.0.0.1:34874)
0806 01:12:17.360488 (+ 1824us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.360502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.360641 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.361221 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.361226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.361252 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.361656 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.361671 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.363703 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.363709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.365342 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.365355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.367786 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.367795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.368155 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.368187 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.368268 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.368387 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1721,"thread_start_us":1601,"threads_started":1}
I20260806 01:12:17.371649 20250 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (DEFAULT_TABLE table=test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:17.371649 20445 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (DEFAULT_TABLE table=test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:17.372295 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.358366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34874 (local address 127.22.63.65:40259)
0806 01:12:17.360054 (+ 1688us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.360066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.360608 (+ 542us) server_negotiation.cc:407] Connection header received
0806 01:12:17.360799 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.360811 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.361002 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.361206 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.361848 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.363488 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.365495 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.367512 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.367662 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.371696 (+ 4034us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.371801 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.371971 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1286,"threads_started":1}
I20260806 01:12:17.373214 20250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:17.373212 20445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:17.375006 20061 tablet_service.cc:1467] Processing CreateTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (DEFAULT_TABLE table=test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:17.376129 20061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:17.387130 20570 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Bootstrap starting.
I20260806 01:12:17.390331 20570 tablet_bootstrap.cc:654] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:17.391237 20571 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Bootstrap starting.
I20260806 01:12:17.391780 20570 log.cc:824] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:17.392221 20572 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Bootstrap starting.
I20260806 01:12:17.395356 20571 tablet_bootstrap.cc:654] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:17.396322 20572 tablet_bootstrap.cc:654] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:17.396689 20571 log.cc:824] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:17.397536 20572 log.cc:824] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:17.398587 20570 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: No bootstrap required, opened a new log
I20260806 01:12:17.398872 20570 ts_tablet_manager.cc:1397] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260806 01:12:17.399643 20571 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: No bootstrap required, opened a new log
I20260806 01:12:17.399919 20571 ts_tablet_manager.cc:1397] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260806 01:12:17.400692 20572 tablet_bootstrap.cc:492] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: No bootstrap required, opened a new log
I20260806 01:12:17.400950 20572 ts_tablet_manager.cc:1397] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 01:12:17.407227 20571 raft_consensus.cc:348] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.407605 20571 raft_consensus.cc:374] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:17.407727 20571 raft_consensus.cc:729] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b700f6f572c64396a957bd24797766f8, State: Initialized, Role: FOLLOWER
I20260806 01:12:17.408344 20571 consensus_queue.cc:260] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.409399 20351 heartbeater.cc:503] Master 127.22.63.126:40515 was elected leader, sending a full tablet report...
I20260806 01:12:17.409642 20570 raft_consensus.cc:348] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.410027 20570 raft_consensus.cc:374] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:17.410084 20571 ts_tablet_manager.cc:1428] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260806 01:12:17.410108 20570 raft_consensus.cc:729] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e290ae07dd3f47b69097b102135b1fab, State: Initialized, Role: FOLLOWER
I20260806 01:12:17.410655 20570 consensus_queue.cc:260] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.411442 20572 raft_consensus.cc:348] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.411868 20572 raft_consensus.cc:374] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:17.411973 20572 raft_consensus.cc:729] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9fc3c5822e4ca1ae75e08eae4c89fa, State: Initialized, Role: FOLLOWER
I20260806 01:12:17.412405 20570 ts_tablet_manager.cc:1428] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260806 01:12:17.412868 20572 consensus_queue.cc:260] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.414084 20545 heartbeater.cc:503] Master 127.22.63.126:40515 was elected leader, sending a full tablet report...
I20260806 01:12:17.414695 20572 ts_tablet_manager.cc:1428] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260806 01:12:17.422886 20154 tablet.cc:2369] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:12:17.477573 20354 tablet.cc:2369] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:12:17.535316 20546 tablet.cc:2369] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:17.561421 20578 raft_consensus.cc:482] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:17.561774 20578 raft_consensus.cc:504] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.563625 20578 leader_election.cc:288] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700f6f572c64396a957bd24797766f8 (127.22.63.66:36921), e290ae07dd3f47b69097b102135b1fab (127.22.63.65:40259)
I20260806 01:12:17.565196 20577 raft_consensus.cc:482] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:17.565632 20577 raft_consensus.cc:504] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.574492 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.565030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:39485 (local address 127.22.63.65:40259)
0806 01:12:17.565235 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.565241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.566043 (+ 802us) server_negotiation.cc:407] Connection header received
0806 01:12:17.566190 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.566198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.566290 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.566406 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.567081 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.568093 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.571199 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.572856 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.573038 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.573161 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.573929 (+ 768us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.574130 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 01:12:17.574427 20577 leader_election.cc:288] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b700f6f572c64396a957bd24797766f8 (127.22.63.66:36921), 2f9fc3c5822e4ca1ae75e08eae4c89fa (127.22.63.67:43541)
I20260806 01:12:17.575433 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.564716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:40259 (local address 127.22.63.67:39485)
0806 01:12:17.565912 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.565933 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.566067 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.566449 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.566453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.566480 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.566881 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.566896 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.568317 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.568323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.570976 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.570998 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.573384 (+ 2386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.573393 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.573665 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.573723 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.574719 (+ 996us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.574927 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"mutex_wait_us":9,"thread_start_us":738,"threads_started":1}
I20260806 01:12:17.578238 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290ae07dd3f47b69097b102135b1fab" is_pre_election: true
I20260806 01:12:17.578902 20085 raft_consensus.cc:2454] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9fc3c5822e4ca1ae75e08eae4c89fa in term 0.
I20260806 01:12:17.579666 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.564130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36921 (local address 127.22.63.67:40847)
0806 01:12:17.564422 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.564447 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.564619 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.565353 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.565362 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.565460 (+ 98us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.565749 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.565774 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.567165 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.567172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.569556 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.569578 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.578086 (+ 8508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.578095 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.578355 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.578410 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.578523 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.579274 (+ 751us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":58}
I20260806 01:12:17.580260 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.564261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:40847 (local address 127.22.63.66:36921)
0806 01:12:17.564483 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.564491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.564515 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:12:17.564840 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.564848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.564962 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.565101 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.565932 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.567020 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.569813 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.571672 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.577897 (+ 6225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.577996 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.578784 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.578988 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:12:17.579802 20381 leader_election.cc:302] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f9fc3c5822e4ca1ae75e08eae4c89fa, e290ae07dd3f47b69097b102135b1fab; no voters:
I20260806 01:12:17.580856 20578 raft_consensus.cc:2781] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:17.581027 20578 raft_consensus.cc:482] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:17.581156 20578 raft_consensus.cc:3037] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:17.584650 20578 raft_consensus.cc:504] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.585613 20578 leader_election.cc:288] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [CANDIDATE]: Term 1 election: Requested vote from peers b700f6f572c64396a957bd24797766f8 (127.22.63.66:36921), e290ae07dd3f47b69097b102135b1fab (127.22.63.65:40259)
I20260806 01:12:17.586395 20085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e290ae07dd3f47b69097b102135b1fab"
I20260806 01:12:17.586642 20085 raft_consensus.cc:3037] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:17.587060 20280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700f6f572c64396a957bd24797766f8"
I20260806 01:12:17.587080 20281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700f6f572c64396a957bd24797766f8" is_pre_election: true
I20260806 01:12:17.587709 20281 raft_consensus.cc:2454] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f9fc3c5822e4ca1ae75e08eae4c89fa in term 0.
I20260806 01:12:17.588171 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.575189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:54719 (local address 127.22.63.67:43541)
0806 01:12:17.575453 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.575462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.577109 (+ 1647us) server_negotiation.cc:407] Connection header received
0806 01:12:17.577429 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.577436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.577523 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.577594 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.580706 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.582382 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.585364 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.586805 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.586986 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.587084 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.587781 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.587916 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:12:17.588935 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.568179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:43541 (local address 127.22.63.65:54719)
0806 01:12:17.576984 (+ 8805us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.577012 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.577250 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.580079 (+ 2829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.580084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.580116 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.580508 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.580524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.582599 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.582608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.585180 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.585201 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.587297 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.587307 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.587507 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.587547 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.588330 (+ 783us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.588491 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8634,"thread_start_us":8916,"threads_started":1}
I20260806 01:12:17.589318 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.568282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:36921 (local address 127.22.63.65:50103)
0806 01:12:17.577415 (+ 9133us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:17.577436 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:17.577558 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:17.578323 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:17.578328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:17.578361 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:17.578828 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.578843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.580599 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.580605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.583049 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:17.583073 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.585116 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.585125 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:17.585294 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.585344 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 01:12:17.585432 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 01:12:17.585569 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8986,"thread_start_us":8292,"threads_started":1}
I20260806 01:12:17.589764 20085 raft_consensus.cc:2454] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f9fc3c5822e4ca1ae75e08eae4c89fa in term 1.
I20260806 01:12:17.589833 20481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "e290ae07dd3f47b69097b102135b1fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" is_pre_election: true
I20260806 01:12:17.590152 20381 leader_election.cc:302] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2f9fc3c5822e4ca1ae75e08eae4c89fa, e290ae07dd3f47b69097b102135b1fab; no voters: b700f6f572c64396a957bd24797766f8
I20260806 01:12:17.590332 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.568894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:50103 (local address 127.22.63.66:36921)
0806 01:12:17.576394 (+ 7500us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.576403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.577719 (+ 1316us) server_negotiation.cc:407] Connection header received
0806 01:12:17.577813 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.577818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.577899 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.577999 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:17.579088 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.580464 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.583274 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.584647 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.584840 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.589727 (+ 4887us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.589788 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.589950 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7298,"thread_start_us":7213,"threads_started":1}
I20260806 01:12:17.590422 20481 raft_consensus.cc:2379] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e290ae07dd3f47b69097b102135b1fab in current term 1: Already voted for candidate 2f9fc3c5822e4ca1ae75e08eae4c89fa in this term.
I20260806 01:12:17.590653 20578 raft_consensus.cc:2781] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:17.590986 20280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e3f6fe9c1cc4feeb28d49501f9000fd" candidate_uuid: "e290ae07dd3f47b69097b102135b1fab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b700f6f572c64396a957bd24797766f8" is_pre_election: true
I20260806 01:12:17.591285 20280 raft_consensus.cc:2454] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e290ae07dd3f47b69097b102135b1fab in term 0.
I20260806 01:12:17.591794 20578 raft_consensus.cc:686] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 LEADER]: Becoming Leader. State: Replica: 2f9fc3c5822e4ca1ae75e08eae4c89fa, State: Running, Role: LEADER
I20260806 01:12:17.591861 19995 leader_election.cc:302] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b700f6f572c64396a957bd24797766f8, e290ae07dd3f47b69097b102135b1fab; no voters: 2f9fc3c5822e4ca1ae75e08eae4c89fa
I20260806 01:12:17.592337 20577 raft_consensus.cc:2741] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 01:12:17.592331 20578 consensus_queue.cc:237] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } }
I20260806 01:12:17.597532 19868 catalog_manager.cc:5495] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f9fc3c5822e4ca1ae75e08eae4c89fa (127.22.63.67). New cstate: current_term: 1 leader_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f9fc3c5822e4ca1ae75e08eae4c89fa" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 43541 } health_report { overall_health: HEALTHY } } }
I20260806 01:12:17.660380 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.655067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.22.63.67:43541)
0806 01:12:17.655248 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.655252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.655269 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:12:17.655361 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.655365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.655446 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.655579 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:17.655976 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.656768 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.658180 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.658585 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.658722 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.659219 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:17.659764 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:17.659923 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.659994 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.660080 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:12:17.668802 20280 raft_consensus.cc:3037] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:17.668985 20085 raft_consensus.cc:1261] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 1 FOLLOWER]: Refusing update from remote peer 2f9fc3c5822e4ca1ae75e08eae4c89fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:17.669821 20583 consensus_queue.cc:1035] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e290ae07dd3f47b69097b102135b1fab" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 40259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:17.671439 20280 raft_consensus.cc:1261] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 1 FOLLOWER]: Refusing update from remote peer 2f9fc3c5822e4ca1ae75e08eae4c89fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:17.672267 20583 consensus_queue.cc:1035] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b700f6f572c64396a957bd24797766f8" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 36921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:17.695322 19870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:17.695667 19870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:17.699918 19870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bce9fd22453146abb816e4c4c0dfa87f: Sending DeleteTablet for 3 replicas of tablet 4e3f6fe9c1cc4feeb28d49501f9000fd
I20260806 01:12:17.701192 20445 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:17 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:60504
I20260806 01:12:17.701203 20250 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:17 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:46308
I20260806 01:12:17.702190 20061 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e3f6fe9c1cc4feeb28d49501f9000fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:17 UTC) from {username='kudu', principal='kudu/127.22.63.126@KRBTEST.COM'} at 127.0.0.1:34874
I20260806 01:12:17.702337 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19970
I20260806 01:12:17.718605 20601 tablet_replica.cc:330] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: stopping tablet replica
I20260806 01:12:17.719257 20601 raft_consensus.cc:2229] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:17.720592 20600 tablet_replica.cc:330] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: stopping tablet replica
I20260806 01:12:17.720883 20599 tablet_replica.cc:330] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: stopping tablet replica
I20260806 01:12:17.721276 20600 raft_consensus.cc:2229] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:17.721320 20599 raft_consensus.cc:2229] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:17.721760 20600 raft_consensus.cc:2258] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:17.721982 20599 raft_consensus.cc:2258] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:17.722272 20601 raft_consensus.cc:2258] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:17.724462 20599 ts_tablet_manager.cc:1905] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:17.726094 20600 ts_tablet_manager.cc:1905] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:17.732203 20599 ts_tablet_manager.cc:1918] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:17.732385 20599 log.cc:1196] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/4e3f6fe9c1cc4feeb28d49501f9000fd
I20260806 01:12:17.733013 20599 ts_tablet_manager.cc:1939] T 4e3f6fe9c1cc4feeb28d49501f9000fd P 2f9fc3c5822e4ca1ae75e08eae4c89fa: Deleting consensus metadata
I20260806 01:12:17.733350 20600 ts_tablet_manager.cc:1918] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:17.733510 20600 log.cc:1196] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/4e3f6fe9c1cc4feeb28d49501f9000fd
I20260806 01:12:17.735311 19849 catalog_manager.cc:4841] TS 2f9fc3c5822e4ca1ae75e08eae4c89fa (127.22.63.67:43541): tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (table test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]) successfully deleted
I20260806 01:12:17.736163 20600 ts_tablet_manager.cc:1939] T 4e3f6fe9c1cc4feeb28d49501f9000fd P b700f6f572c64396a957bd24797766f8: Deleting consensus metadata
I20260806 01:12:17.736647 20601 ts_tablet_manager.cc:1905] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:17.737953 19850 catalog_manager.cc:4841] TS b700f6f572c64396a957bd24797766f8 (127.22.63.66:36921): tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (table test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]) successfully deleted
Aug 06 01:12:17 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.65@KRBTEST.COM
I20260806 01:12:17.744976 20601 ts_tablet_manager.cc:1918] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:17.745239 20601 log.cc:1196] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785978402713339-22781-0/minicluster-data/ts-0/wal/wals/4e3f6fe9c1cc4feeb28d49501f9000fd
I20260806 01:12:17.745877 20601 ts_tablet_manager.cc:1939] T 4e3f6fe9c1cc4feeb28d49501f9000fd P e290ae07dd3f47b69097b102135b1fab: Deleting consensus metadata
I20260806 01:12:17.747885 19850 catalog_manager.cc:4841] TS e290ae07dd3f47b69097b102135b1fab (127.22.63.65:40259): tablet 4e3f6fe9c1cc4feeb28d49501f9000fd (table test-table [id=b7fc11d9cf5445a49fa8a83db2b35fea]) successfully deleted
I20260806 01:12:17.749915 20567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.704087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.22.63.65:40259)
0806 01:12:17.704317 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.704326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.714764 (+ 10438us) server_negotiation.cc:407] Connection header received
0806 01:12:17.715354 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.715364 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.715506 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.715605 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:17.721018 (+ 5413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.722211 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.734020 (+ 11809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.734683 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.734802 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.741853 (+ 7051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:17.741874 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:17.741895 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:17.741948 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:17.746273 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.747302 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.747311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.747314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.747487 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.748066 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.748071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.748074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.748567 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:17.748775 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.749298 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.749537 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:12:17.750725 20146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:17.842780 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19970
I20260806 01:12:17.876515 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 20159
Aug 06 01:12:17 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.66@KRBTEST.COM
I20260806 01:12:17.897953 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:17.878630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.22.63.66:36921)
0806 01:12:17.878870 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:17.878878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:17.878900 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:12:17.878972 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:17.878977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:17.879061 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:17.879179 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:17.881251 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.882365 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.884505 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:17.885231 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:17.885368 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:17.890699 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:17.890717 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:17.890739 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:17.890788 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:17.894770 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.895575 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.895585 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.895589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.895778 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:17.896147 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:17.896152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:17.896156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:17.896678 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:17.897106 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:17.897295 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 01:12:17.897574 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:12:17.898705 20345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:17.992599 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 20159
I20260806 01:12:18.026412 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 20359
Aug 06 01:12:18 dist-test-slave-7f01 krb5kdc[19804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.67@KRBTEST.COM
I20260806 01:12:18.042917 20566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:18.027338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.22.63.67:43541)
0806 01:12:18.027581 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:18.027589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:18.027613 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:12:18.027682 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:18.027688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:18.027801 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:18.027929 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:18.029199 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:18.029965 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:18.031274 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:18.031877 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:18.032012 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:18.036557 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:18.036576 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:18.036593 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:18.036639 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:18.039950 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:18.040712 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:18.040725 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:18.040727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:18.040881 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:18.041277 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:18.041282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:18.041290 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:18.041694 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:18.042041 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:18.042309 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 01:12:18.042545 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:12:18.043802 20537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:18.168780 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 20359
I20260806 01:12:18.197661 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 19825
I20260806 01:12:18.198962 19917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:18.310307 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 19825
2026-08-06T01:12:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4961 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T01:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:18Z Disabled control of system clock
I20260806 01:12:18.408917 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41563
--webserver_interface=127.22.63.126
--webserver_port=0
--builtin_ntp_servers=127.22.63.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:41563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:18.759547 20621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:18.759865 20621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:18.759959 20621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:18.767185 20621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:18.767302 20621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:18.767341 20621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:18.767416 20621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:18.767460 20621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:18.776698 20621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:41563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:41563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:18.777971 20621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:18.779844 20621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:18.791750 20627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:18.791791 20630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:18.792261 20626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:18.793270 20621 server_base.cc:1034] running on GCE node
I20260806 01:12:18.794108 20621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:18.795591 20621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:18.796901 20621 hybrid_clock.cc:648] HybridClock initialized: now 1785978738796839 us; error 69 us; skew 500 ppm
I20260806 01:12:18.797333 20621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:18.799966 20621 webserver.cc:460] Webserver started at http://127.22.63.126:40713/ using document root <none> and password file <none>
I20260806 01:12:18.800734 20621 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:18.800890 20621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:18.801249 20621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:18.804491 20621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "5f4f84a5b38448e7aefa72cfebac0b21"
format_stamp: "Formatted at 2026-08-06 01:12:18 on dist-test-slave-7f01"
server_key: "f4d64e2d72a899affe8a40c1b484c119"
server_key_iv: "1ae49bdc04e22c59cbbc1a95203d8bce"
server_key_version: "encryptionkey@0"
I20260806 01:12:18.805395 20621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "5f4f84a5b38448e7aefa72cfebac0b21"
format_stamp: "Formatted at 2026-08-06 01:12:18 on dist-test-slave-7f01"
server_key: "f4d64e2d72a899affe8a40c1b484c119"
server_key_iv: "1ae49bdc04e22c59cbbc1a95203d8bce"
server_key_version: "encryptionkey@0"
I20260806 01:12:18.811146 20621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:12:18.814989 20637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:18.816466 20621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:18.816679 20621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "5f4f84a5b38448e7aefa72cfebac0b21"
format_stamp: "Formatted at 2026-08-06 01:12:18 on dist-test-slave-7f01"
server_key: "f4d64e2d72a899affe8a40c1b484c119"
server_key_iv: "1ae49bdc04e22c59cbbc1a95203d8bce"
server_key_version: "encryptionkey@0"
I20260806 01:12:18.816947 20621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:18.848642 20621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:18.849646 20621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:18.849982 20621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:18.871193 20621 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:41563
I20260806 01:12:18.871212 20706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:41563 every 8 connection(s)
I20260806 01:12:18.872850 20621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:18.877036 20707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:18.881268 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 20621
I20260806 01:12:18.881482 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:18.882113 22781 external_mini_cluster.cc:1428] Setting key defc64075882b385d4a06aeb9eaeeb33
I20260806 01:12:18.890765 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: Bootstrap starting.
I20260806 01:12:18.894095 20707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:18.895310 20707 log.cc:824] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:18.896407 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:18.884428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46220 (local address 127.22.63.126:41563)
0806 01:12:18.885458 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:18.885469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:18.885512 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:12:18.887177 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:18.887199 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:18.887678 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:18.888175 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:18.888775 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:18.890062 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:18.891525 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:18.892276 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:18.892485 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:18.893666 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:18.893674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:18.893687 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:18.894264 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:18.894381 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:18.894404 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:18.894533 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:18.894760 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 01:12:18.895063 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":404,"threads_started":1}
I20260806 01:12:18.898640 20707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: No bootstrap required, opened a new log
I20260806 01:12:18.906248 20707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } }
I20260806 01:12:18.906644 20707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:18.906752 20707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4f84a5b38448e7aefa72cfebac0b21, State: Initialized, Role: FOLLOWER
I20260806 01:12:18.907302 20707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } }
I20260806 01:12:18.907558 20707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:18.907701 20707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:18.907902 20707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:18.910044 20707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } }
I20260806 01:12:18.910567 20707 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f4f84a5b38448e7aefa72cfebac0b21; no voters:
I20260806 01:12:18.911152 20707 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:18.911350 20715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:18.912628 20715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4f84a5b38448e7aefa72cfebac0b21, State: Running, Role: LEADER
I20260806 01:12:18.913177 20715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } }
I20260806 01:12:18.913537 20707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:18.916051 20716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } } }
I20260806 01:12:18.916462 20716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:18.917932 20718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4f84a5b38448e7aefa72cfebac0b21. Latest consensus state: current_term: 1 leader_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4f84a5b38448e7aefa72cfebac0b21" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 41563 } } }
I20260806 01:12:18.918224 20718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:18.919533 20726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:18.924405 20726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:18.933509 20726 catalog_manager.cc:1347] Generated new cluster ID: 263dfe610f8e4a9696b1d7f731afe8bd
I20260806 01:12:18.933635 20726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:18.950508 20726 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:18.952452 20726 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:18.959218 20726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: Generated new TSK 0
I20260806 01:12:18.960291 20726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:18.966696 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.65:0
--local_ip_for_outbound_sockets=127.22.63.65
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--builtin_ntp_servers=127.22.63.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 01:12:19.323433 20741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:19.323746 20741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:19.323915 20741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:19.331149 20741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:19.331284 20741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:19.331444 20741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.65
I20260806 01:12:19.340682 20741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.63.65
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.65
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:19.341997 20741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:19.343971 20741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:19.357627 20751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:19.357659 20749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:19.357702 20748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:19.363245 20741 server_base.cc:1034] running on GCE node
I20260806 01:12:19.363992 20741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:19.365002 20741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:19.366250 20741 hybrid_clock.cc:648] HybridClock initialized: now 1785978739366206 us; error 55 us; skew 500 ppm
I20260806 01:12:19.366581 20741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:19.369025 20741 webserver.cc:460] Webserver started at http://127.22.63.65:42227/ using document root <none> and password file <none>
I20260806 01:12:19.369756 20741 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:19.369891 20741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:19.370223 20741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:19.373196 20741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data/instance:
uuid: "1bdf219b25874cc5936365a0b6972430"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "5d2b0bc4f8a73f9de855e70f31733213"
server_key_iv: "1f7d3b434e5f9053e85fa72cc3d3f67a"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.373996 20741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance:
uuid: "1bdf219b25874cc5936365a0b6972430"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "5d2b0bc4f8a73f9de855e70f31733213"
server_key_iv: "1f7d3b434e5f9053e85fa72cc3d3f67a"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.379125 20741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:12:19.382575 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.383910 20741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:19.384096 20741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal
uuid: "1bdf219b25874cc5936365a0b6972430"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "5d2b0bc4f8a73f9de855e70f31733213"
server_key_iv: "1f7d3b434e5f9053e85fa72cc3d3f67a"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.384320 20741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:19.411827 20741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:19.412721 20741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:19.413049 20741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:19.414171 20741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:19.415781 20741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:19.415904 20741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.416040 20741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:19.416124 20741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.451100 20741 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.65:43247
I20260806 01:12:19.451165 20929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.65:43247 every 8 connection(s)
I20260806 01:12:19.452486 20741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/data/info.pb
I20260806 01:12:19.453480 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 20741
I20260806 01:12:19.453671 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-0/wal/instance
I20260806 01:12:19.454221 22781 external_mini_cluster.cc:1428] Setting key 770121eed28d15b7c27fcd251b591839
I20260806 01:12:19.457532 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.66:0
--local_ip_for_outbound_sockets=127.22.63.66
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--builtin_ntp_servers=127.22.63.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:19.465978 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:19.455308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:56993 (local address 127.22.63.126:41563)
0806 01:12:19.455848 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:19.455854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:19.457199 (+ 1345us) server_negotiation.cc:407] Connection header received
0806 01:12:19.459271 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:19.459276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:19.459373 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:19.459531 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:19.460634 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.461353 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.463096 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.463526 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.463662 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:19.465096 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:19.465101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:19.465104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:19.465124 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:19.465166 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:19.465178 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:19.465269 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:19.465509 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 01:12:19.465609 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":273,"threads_started":1}
I20260806 01:12:19.466974 20936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:19.456019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41563 (local address 127.22.63.65:56993)
0806 01:12:19.457050 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:19.457097 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:19.459013 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:19.459791 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:19.459804 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:19.459896 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:19.460466 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:19.460483 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.461525 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.461530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:19.462944 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:19.462956 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.463788 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.463796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:19.464022 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:19.464745 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:19.464747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:19.464787 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:19.464862 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:19.465284 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:19.465297 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:19.465309 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 01:12:19.465477 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 01:12:19.465857 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":370,"threads_started":1}
I20260806 01:12:19.468606 20930 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41563
I20260806 01:12:19.468916 20930 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:19.469648 20930 heartbeater.cc:511] Master 127.22.63.126:41563 requested a full tablet report, sending...
I20260806 01:12:19.471951 20662 ts_manager.cc:194] Registered new tserver with Master: 1bdf219b25874cc5936365a0b6972430 (127.22.63.65:43247)
I20260806 01:12:19.474171 20662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.65:56993
W20260806 01:12:19.812633 20939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:19.812915 20939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:19.813004 20939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:19.820421 20939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:19.820518 20939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:19.820626 20939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.66
I20260806 01:12:19.829538 20939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.63.66
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.66
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:19.830771 20939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:19.832774 20939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:19.845839 20947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:19.846004 20946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:19.845979 20953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:19.847268 20939 server_base.cc:1034] running on GCE node
I20260806 01:12:19.847992 20939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:19.848945 20939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:19.850152 20939 hybrid_clock.cc:648] HybridClock initialized: now 1785978739850107 us; error 32 us; skew 500 ppm
I20260806 01:12:19.850477 20939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:19.855955 20939 webserver.cc:460] Webserver started at http://127.22.63.66:43817/ using document root <none> and password file <none>
I20260806 01:12:19.856689 20939 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:19.856827 20939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:19.857167 20939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:19.860174 20939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data/instance:
uuid: "1d1eee089049462e905df026d26a382a"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "1d9caa85700bd5edff1d81776c2407fe"
server_key_iv: "7de332ec1960bb070a3e07647bfc634c"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.860997 20939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance:
uuid: "1d1eee089049462e905df026d26a382a"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "1d9caa85700bd5edff1d81776c2407fe"
server_key_iv: "7de332ec1960bb070a3e07647bfc634c"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.866292 20939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 01:12:19.869951 20960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.871580 20939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:19.871834 20939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal
uuid: "1d1eee089049462e905df026d26a382a"
format_stamp: "Formatted at 2026-08-06 01:12:19 on dist-test-slave-7f01"
server_key: "1d9caa85700bd5edff1d81776c2407fe"
server_key_iv: "7de332ec1960bb070a3e07647bfc634c"
server_key_version: "encryptionkey@0"
I20260806 01:12:19.872092 20939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:19.897641 20939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:19.898466 20939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:19.898732 20939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:19.899672 20939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:19.901240 20939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:19.901360 20939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.901489 20939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:19.901572 20939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:19.935847 20939 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.66:37545
I20260806 01:12:19.935863 21120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.66:37545 every 8 connection(s)
I20260806 01:12:19.937211 20939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/data/info.pb
I20260806 01:12:19.943688 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 20939
I20260806 01:12:19.943897 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal/instance
I20260806 01:12:19.944479 22781 external_mini_cluster.cc:1428] Setting key 37b680af5a21ffc7d537ab5d460e2dd4
I20260806 01:12:19.947425 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.63.67:0
--local_ip_for_outbound_sockets=127.22.63.67
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--builtin_ntp_servers=127.22.63.84:45161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 01:12:19.950656 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:19.939960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.66:40387 (local address 127.22.63.126:41563)
0806 01:12:19.940195 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:19.940202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:19.941656 (+ 1454us) server_negotiation.cc:407] Connection header received
0806 01:12:19.943437 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:19.943442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:19.943515 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:19.943611 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:19.945059 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.945959 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.947799 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.948206 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.948345 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:19.949698 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:19.949703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:19.949707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:19.949724 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:19.949763 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:19.949774 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:19.949874 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:19.950229 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 01:12:19.950343 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 01:12:19.951658 21125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:19.940557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41563 (local address 127.22.63.66:40387)
0806 01:12:19.941488 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:19.941530 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:19.943127 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:19.943907 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:19.943920 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:19.944043 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:19.944854 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:19.944880 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.946112 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.946119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:19.947624 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:19.947644 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:19.948441 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:19.948449 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:19.948683 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:19.949392 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:19.949394 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:19.949433 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:19.949510 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:19.949966 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:19.949986 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:19.949993 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 01:12:19.950206 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 01:12:19.950592 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":314,"threads_started":1}
I20260806 01:12:19.953177 21121 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41563
I20260806 01:12:19.953476 21121 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:19.954252 21121 heartbeater.cc:511] Master 127.22.63.126:41563 requested a full tablet report, sending...
I20260806 01:12:19.955761 20662 ts_manager.cc:194] Registered new tserver with Master: 1d1eee089049462e905df026d26a382a (127.22.63.66:37545)
I20260806 01:12:19.957002 20662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.66:40387
W20260806 01:12:20.302821 21126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:20.303118 21126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:20.303269 21126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:20.310644 21126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:20.310756 21126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:20.310942 21126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.63.67
I20260806 01:12:20.320597 21126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:45161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.63.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.63.67
--webserver_port=0
--tserver_master_addrs=127.22.63.126:41563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.63.67
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:20.322065 21126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:20.323925 21126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:20.337546 21132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:20.337587 21133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:20.337546 21137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:20.341173 21126 server_base.cc:1034] running on GCE node
I20260806 01:12:20.341814 21126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:20.342854 21126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:20.344077 21126 hybrid_clock.cc:648] HybridClock initialized: now 1785978740344034 us; error 40 us; skew 500 ppm
I20260806 01:12:20.344405 21126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 01:12:20.346802 21126 webserver.cc:460] Webserver started at http://127.22.63.67:39037/ using document root <none> and password file <none>
I20260806 01:12:20.347539 21126 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:20.347671 21126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:20.348042 21126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:20.351049 21126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data/instance:
uuid: "53d7d36fb01e46738e3cd140334dcff7"
format_stamp: "Formatted at 2026-08-06 01:12:20 on dist-test-slave-7f01"
server_key: "7a45a7e4fb67e8555346dd7fe1efa008"
server_key_iv: "47bf2ac7fc6b4881ccfddecf1c6eb7e5"
server_key_version: "encryptionkey@0"
I20260806 01:12:20.352058 21126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance:
uuid: "53d7d36fb01e46738e3cd140334dcff7"
format_stamp: "Formatted at 2026-08-06 01:12:20 on dist-test-slave-7f01"
server_key: "7a45a7e4fb67e8555346dd7fe1efa008"
server_key_iv: "47bf2ac7fc6b4881ccfddecf1c6eb7e5"
server_key_version: "encryptionkey@0"
I20260806 01:12:20.357307 21126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:12:20.360886 21150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:20.362195 21126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 01:12:20.362380 21126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal
uuid: "53d7d36fb01e46738e3cd140334dcff7"
format_stamp: "Formatted at 2026-08-06 01:12:20 on dist-test-slave-7f01"
server_key: "7a45a7e4fb67e8555346dd7fe1efa008"
server_key_iv: "47bf2ac7fc6b4881ccfddecf1c6eb7e5"
server_key_version: "encryptionkey@0"
I20260806 01:12:20.362608 21126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:20.384301 21126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:20.385181 21126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:20.385506 21126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:20.386528 21126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 01:12:20.388190 21126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 01:12:20.388316 21126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:20.388437 21126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 01:12:20.388528 21126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:20.422765 21126 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.67:35177
I20260806 01:12:20.422859 21325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.67:35177 every 8 connection(s)
I20260806 01:12:20.424136 21126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/data/info.pb
I20260806 01:12:20.432898 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 21126
I20260806 01:12:20.433095 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal/instance
I20260806 01:12:20.433565 22781 external_mini_cluster.cc:1428] Setting key 506f8dced14dc27f796cf755cbc58a22
I20260806 01:12:20.436710 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.426636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.67:54441 (local address 127.22.63.126:41563)
0806 01:12:20.426826 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.426832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.428259 (+ 1427us) server_negotiation.cc:407] Connection header received
0806 01:12:20.430008 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.430013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.430085 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.430177 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:20.431160 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.431965 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.433592 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.434156 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.434290 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.435857 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:20.435863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:20.435866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:20.435884 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:20.435916 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:20.435928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:20.436002 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.436270 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.436351 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:12:20.437744 21331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.427187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:41563 (local address 127.22.63.67:54441)
0806 01:12:20.428103 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.428147 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.429757 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.430370 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.430377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.430455 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:12:20.431010 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.431026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.432104 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.432108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.433449 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.433461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.434383 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.434389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.434595 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.435446 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 01:12:20.435449 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:12:20.435506 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 01:12:20.435589 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:12:20.436040 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:12:20.436054 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:12:20.436062 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.436262 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.436704 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260806 01:12:20.439209 21329 heartbeater.cc:347] Connected to a master server at 127.22.63.126:41563
I20260806 01:12:20.439549 21329 heartbeater.cc:464] Registering TS with master...
I20260806 01:12:20.440380 21329 heartbeater.cc:511] Master 127.22.63.126:41563 requested a full tablet report, sending...
I20260806 01:12:20.441728 20662 ts_manager.cc:194] Registered new tserver with Master: 53d7d36fb01e46738e3cd140334dcff7 (127.22.63.67:35177)
I20260806 01:12:20.442576 20662 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.63.67:54441
I20260806 01:12:20.449050 22781 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 01:12:20.459714 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.453346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46230 (local address 127.22.63.126:41563)
0806 01:12:20.453587 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.453596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.453997 (+ 401us) server_negotiation.cc:407] Connection header received
0806 01:12:20.454185 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.454188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.454258 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.454348 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:20.455085 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.455835 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.457500 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.457915 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.458028 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.459037 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:20.459042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:20.459045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:20.459062 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:20.459092 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:20.459104 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:20.459176 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.459311 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.459385 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:12:20.463624 20662 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 01:12:20.470577 20662 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 01:12:20.478081 20930 heartbeater.cc:503] Master 127.22.63.126:41563 was elected leader, sending a full tablet report...
I20260806 01:12:20.507668 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.483067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.22.63.66:37545)
0806 01:12:20.497069 (+ 14002us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.497080 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.497116 (+ 36us) server_negotiation.cc:407] Connection header received
0806 01:12:20.497232 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.497243 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.497537 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.498836 (+ 1299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:20.499142 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.501219 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.504982 (+ 3763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.506409 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.506574 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.506864 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.507104 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.507279 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13826,"thread_start_us":13691,"threads_started":1}
I20260806 01:12:20.507682 21351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.482952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:37545 (local address 127.0.0.1:53268)
0806 01:12:20.494457 (+ 11505us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.494537 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.494818 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.497794 (+ 2976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.497805 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.497842 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:20.498520 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.498545 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.501427 (+ 2882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.501436 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.504719 (+ 3283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.504733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.506621 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.506627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.506842 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.506899 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.507067 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.507261 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11083,"mutex_wait_us":161,"thread_start_us":14293,"threads_started":1}
I20260806 01:12:20.508117 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.483123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:35177 (local address 127.0.0.1:54276)
0806 01:12:20.496991 (+ 13868us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.497010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.497166 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.498132 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.498139 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.498170 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:20.498790 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.498809 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.501542 (+ 2733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.501550 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.505010 (+ 3460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.505026 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.507291 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.507299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.507490 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.507523 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.507609 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.507750 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13745,"mutex_wait_us":107,"thread_start_us":11981,"threads_started":1}
I20260806 01:12:20.508944 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.483441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.22.63.67:35177)
0806 01:12:20.496585 (+ 13144us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.496596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.497129 (+ 533us) server_negotiation.cc:407] Connection header received
0806 01:12:20.497326 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.497332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.497522 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.497771 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:20.499084 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.501298 (+ 2214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.505246 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.506704 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.506850 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.507158 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.508385 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.508559 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12886,"thread_start_us":12696,"threads_started":1}
I20260806 01:12:20.512506 21030 tablet_service.cc:1467] Processing CreateTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 (DEFAULT_TABLE table=test-table [id=6dec572705194abbb9b4397807a6390f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:20.512502 21215 tablet_service.cc:1467] Processing CreateTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 (DEFAULT_TABLE table=test-table [id=6dec572705194abbb9b4397807a6390f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:20.514063 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.497693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60122)
0806 01:12:20.504280 (+ 6587us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.504313 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.504424 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.506085 (+ 1661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.506090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.506118 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:20.506506 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.506519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.508951 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.508958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.510959 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.510974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.513246 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.513252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.513443 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.513489 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.513590 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.513753 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6449,"thread_start_us":6807,"threads_started":1}
I20260806 01:12:20.514196 21215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21983c1ae0448a4a8fbc51d9a652aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:20.514376 21030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21983c1ae0448a4a8fbc51d9a652aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:20.514743 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.498065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.22.63.65:43247)
0806 01:12:20.505305 (+ 7240us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.505313 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.505345 (+ 32us) server_negotiation.cc:407] Connection header received
0806 01:12:20.505450 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.505477 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.505738 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.505942 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:20.506744 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.508768 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.511172 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.512687 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.512837 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.513098 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.514062 (+ 964us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.514291 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7047,"thread_start_us":7708,"threads_started":1}
I20260806 01:12:20.518389 20833 tablet_service.cc:1467] Processing CreateTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 (DEFAULT_TABLE table=test-table [id=6dec572705194abbb9b4397807a6390f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 01:12:20.519933 20833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b21983c1ae0448a4a8fbc51d9a652aa5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:20.526692 21356 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Bootstrap starting.
I20260806 01:12:20.530834 21356 tablet_bootstrap.cc:654] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:20.532624 21356 log.cc:824] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:20.533641 21358 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Bootstrap starting.
I20260806 01:12:20.537187 21360 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Bootstrap starting.
I20260806 01:12:20.537634 21356 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: No bootstrap required, opened a new log
I20260806 01:12:20.537941 21356 ts_tablet_manager.cc:1397] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 01:12:20.538034 21358 tablet_bootstrap.cc:654] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:20.539351 21358 log.cc:824] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:20.541360 21360 tablet_bootstrap.cc:654] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:20.542624 21360 log.cc:824] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:20.542717 21358 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: No bootstrap required, opened a new log
I20260806 01:12:20.542994 21358 ts_tablet_manager.cc:1397] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 01:12:20.546838 21356 raft_consensus.cc:348] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.547187 21360 tablet_bootstrap.cc:492] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: No bootstrap required, opened a new log
I20260806 01:12:20.547295 21356 raft_consensus.cc:374] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:20.547441 21360 ts_tablet_manager.cc:1397] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260806 01:12:20.547441 21356 raft_consensus.cc:729] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d1eee089049462e905df026d26a382a, State: Initialized, Role: FOLLOWER
I20260806 01:12:20.548219 21356 consensus_queue.cc:260] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.549453 21121 heartbeater.cc:503] Master 127.22.63.126:41563 was elected leader, sending a full tablet report...
I20260806 01:12:20.550047 21356 ts_tablet_manager.cc:1428] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260806 01:12:20.552129 21358 raft_consensus.cc:348] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.552485 21358 raft_consensus.cc:374] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:20.552600 21358 raft_consensus.cc:729] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d7d36fb01e46738e3cd140334dcff7, State: Initialized, Role: FOLLOWER
I20260806 01:12:20.553108 21358 consensus_queue.cc:260] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.557073 21360 raft_consensus.cc:348] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.557466 21360 raft_consensus.cc:374] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:20.557490 21329 heartbeater.cc:503] Master 127.22.63.126:41563 was elected leader, sending a full tablet report...
I20260806 01:12:20.557576 21360 raft_consensus.cc:729] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bdf219b25874cc5936365a0b6972430, State: Initialized, Role: FOLLOWER
I20260806 01:12:20.558135 21358 ts_tablet_manager.cc:1428] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260806 01:12:20.558176 21360 consensus_queue.cc:260] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.559758 21360 ts_tablet_manager.cc:1428] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
W20260806 01:12:20.675941 21330 tablet.cc:2369] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:12:20.689347 21122 tablet.cc:2369] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 01:12:20.704962 20931 tablet.cc:2369] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:12:20.732457 21366 raft_consensus.cc:482] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 01:12:20.732779 21366 raft_consensus.cc:504] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.734601 21366 leader_election.cc:288] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d1eee089049462e905df026d26a382a (127.22.63.66:37545), 53d7d36fb01e46738e3cd140334dcff7 (127.22.63.67:35177)
I20260806 01:12:20.742415 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.735271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:39603 (local address 127.22.63.67:35177)
0806 01:12:20.735514 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.735523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.736239 (+ 716us) server_negotiation.cc:407] Connection header received
0806 01:12:20.736418 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.736422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.736491 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.736584 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:20.737196 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.737954 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.740161 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.741420 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.741584 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.741653 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.742027 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.742174 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 01:12:20.742533 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.735139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.63.65:56809 (local address 127.22.63.66:37545)
0806 01:12:20.735403 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.735412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.736140 (+ 728us) server_negotiation.cc:407] Connection header received
0806 01:12:20.736418 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.736422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.736498 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.736601 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 01:12:20.737329 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.738032 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.740194 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.741420 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.741580 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.741653 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.742143 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.742266 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:12:20.742522 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.735162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.67:35177 (local address 127.22.63.65:39603)
0806 01:12:20.736128 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.736143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.736258 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.736610 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.736614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.736647 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:20.737008 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.737017 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.738086 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.738093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.739915 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.739927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.741616 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.741623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.741797 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.741871 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.741974 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.742116 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":131,"thread_start_us":589,"threads_started":1}
I20260806 01:12:20.743168 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.734806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.66:37545 (local address 127.22.63.65:56809)
0806 01:12:20.736009 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:20.736027 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 01:12:20.736210 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:12:20.736633 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:12:20.736637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:12:20.736661 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 01:12:20.737180 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.737193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.738172 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.738175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.739991 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:12:20.740010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.741793 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.741797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:12:20.741930 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.741988 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 01:12:20.742806 (+ 818us) client_negotiation.cc:240] Negotiation successful
0806 01:12:20.742937 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":75,"thread_start_us":826,"threads_started":1}
I20260806 01:12:20.743606 21250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21983c1ae0448a4a8fbc51d9a652aa5" candidate_uuid: "1bdf219b25874cc5936365a0b6972430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53d7d36fb01e46738e3cd140334dcff7" is_pre_election: true
I20260806 01:12:20.744196 21250 raft_consensus.cc:2454] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bdf219b25874cc5936365a0b6972430 in term 0.
I20260806 01:12:20.744104 21057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21983c1ae0448a4a8fbc51d9a652aa5" candidate_uuid: "1bdf219b25874cc5936365a0b6972430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1eee089049462e905df026d26a382a" is_pre_election: true
I20260806 01:12:20.744573 21057 raft_consensus.cc:2454] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bdf219b25874cc5936365a0b6972430 in term 0.
I20260806 01:12:20.744867 20764 leader_election.cc:302] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bdf219b25874cc5936365a0b6972430, 53d7d36fb01e46738e3cd140334dcff7; no voters:
I20260806 01:12:20.745285 21366 raft_consensus.cc:2781] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 01:12:20.745452 21366 raft_consensus.cc:482] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 01:12:20.745565 21366 raft_consensus.cc:3037] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:20.747725 21366 raft_consensus.cc:504] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.748425 21366 leader_election.cc:288] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [CANDIDATE]: Term 1 election: Requested vote from peers 1d1eee089049462e905df026d26a382a (127.22.63.66:37545), 53d7d36fb01e46738e3cd140334dcff7 (127.22.63.67:35177)
I20260806 01:12:20.748800 21057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21983c1ae0448a4a8fbc51d9a652aa5" candidate_uuid: "1bdf219b25874cc5936365a0b6972430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d1eee089049462e905df026d26a382a"
I20260806 01:12:20.748924 21250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b21983c1ae0448a4a8fbc51d9a652aa5" candidate_uuid: "1bdf219b25874cc5936365a0b6972430" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53d7d36fb01e46738e3cd140334dcff7"
I20260806 01:12:20.749001 21057 raft_consensus.cc:3037] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:20.749154 21250 raft_consensus.cc:3037] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:20.751483 21250 raft_consensus.cc:2454] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bdf219b25874cc5936365a0b6972430 in term 1.
I20260806 01:12:20.751552 21057 raft_consensus.cc:2454] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bdf219b25874cc5936365a0b6972430 in term 1.
I20260806 01:12:20.751888 20764 leader_election.cc:302] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bdf219b25874cc5936365a0b6972430, 53d7d36fb01e46738e3cd140334dcff7; no voters:
I20260806 01:12:20.752193 21366 raft_consensus.cc:2781] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:20.752646 21366 raft_consensus.cc:686] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 1 LEADER]: Becoming Leader. State: Replica: 1bdf219b25874cc5936365a0b6972430, State: Running, Role: LEADER
I20260806 01:12:20.753125 21366 consensus_queue.cc:237] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } }
I20260806 01:12:20.757814 20662 catalog_manager.cc:5495] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bdf219b25874cc5936365a0b6972430 (127.22.63.65). New cstate: current_term: 1 leader_uuid: "1bdf219b25874cc5936365a0b6972430" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bdf219b25874cc5936365a0b6972430" member_type: VOTER last_known_addr { host: "127.22.63.65" port: 43247 } health_report { overall_health: HEALTHY } } }
I20260806 01:12:20.812101 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.806566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.22.63.65:43247)
0806 01:12:20.806764 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.806779 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.806799 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:12:20.806898 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.806902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.807015 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.807158 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:12:20.807555 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.808415 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.809806 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.810265 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.810379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.810918 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:12:20.811483 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:12:20.811609 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.811732 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.811889 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:12:20.820748 21057 raft_consensus.cc:1261] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 1 FOLLOWER]: Refusing update from remote peer 1bdf219b25874cc5936365a0b6972430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:20.820748 21250 raft_consensus.cc:1261] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 1 FOLLOWER]: Refusing update from remote peer 1bdf219b25874cc5936365a0b6972430: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:12:20.821842 21373 consensus_queue.cc:1035] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d1eee089049462e905df026d26a382a" member_type: VOTER last_known_addr { host: "127.22.63.66" port: 37545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:20.822207 21366 consensus_queue.cc:1035] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53d7d36fb01e46738e3cd140334dcff7" member_type: VOTER last_known_addr { host: "127.22.63.67" port: 35177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:12:20.832378 21377 mvcc.cc:205] Tried to move back new op lower bound from 7315368922390552576 to 7315368922128252928. Current Snapshot: MvccSnapshot[applied={T|T < 7315368922390552576}]
I20260806 01:12:20.835896 21378 mvcc.cc:205] Tried to move back new op lower bound from 7315368922390552576 to 7315368922128252928. Current Snapshot: MvccSnapshot[applied={T|T < 7315368922390552576 or (T in {7315368922390552576})}]
I20260806 01:12:20.853250 20661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:20.853590 20661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 01:12:20.857517 20661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f4f84a5b38448e7aefa72cfebac0b21: Sending DeleteTablet for 3 replicas of tablet b21983c1ae0448a4a8fbc51d9a652aa5
I20260806 01:12:20.858687 21215 tablet_service.cc:1514] Processing DeleteTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:20 UTC) from {username='slave'} at 127.0.0.1:54276
I20260806 01:12:20.858820 20833 tablet_service.cc:1514] Processing DeleteTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:20 UTC) from {username='slave'} at 127.0.0.1:60122
I20260806 01:12:20.858811 21030 tablet_service.cc:1514] Processing DeleteTablet for tablet b21983c1ae0448a4a8fbc51d9a652aa5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 01:12:20 UTC) from {username='slave'} at 127.0.0.1:53268
I20260806 01:12:20.861208 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 20741
I20260806 01:12:20.869027 21391 tablet_replica.cc:330] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: stopping tablet replica
I20260806 01:12:20.869562 21391 raft_consensus.cc:2229] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:20.870139 21391 raft_consensus.cc:2258] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:20.890341 21391 ts_tablet_manager.cc:1905] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:20.896268 21389 tablet_replica.cc:330] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: stopping tablet replica
I20260806 01:12:20.896795 21389 raft_consensus.cc:2229] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 01:12:20.897382 21389 raft_consensus.cc:2258] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:20.902953 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:20.863197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.22.63.65:43247)
0806 01:12:20.863478 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:20.863488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:20.887914 (+ 24426us) server_negotiation.cc:407] Connection header received
0806 01:12:20.888269 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:20.888278 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:20.888448 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:20.888545 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:20.889533 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.891141 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.894871 (+ 3730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:20.895867 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:20.895996 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:20.900594 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:20.900609 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:20.900634 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:20.901680 (+ 1046us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:20.901758 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:20.901805 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:20.901905 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:20.902229 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 01:12:20.902384 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:12:20.904136 21389 ts_tablet_manager.cc:1905] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 01:12:20.904296 21388 tablet_replica.cc:330] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430: stopping tablet replica
I20260806 01:12:20.904848 21388 raft_consensus.cc:2229] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 1 LEADER]: Raft consensus shutting down.
I20260806 01:12:20.904978 20924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:20.906607 21391 ts_tablet_manager.cc:1918] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:20.906838 21391 log.cc:1196] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-1/wal/wals/b21983c1ae0448a4a8fbc51d9a652aa5
I20260806 01:12:20.907464 21391 ts_tablet_manager.cc:1939] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1d1eee089049462e905df026d26a382a: Deleting consensus metadata
I20260806 01:12:20.909497 20641 catalog_manager.cc:4841] TS 1d1eee089049462e905df026d26a382a (127.22.63.66:37545): tablet b21983c1ae0448a4a8fbc51d9a652aa5 (table test-table [id=6dec572705194abbb9b4397807a6390f]) successfully deleted
I20260806 01:12:20.912715 21389 ts_tablet_manager.cc:1918] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 01:12:20.912938 21389 log.cc:1196] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Deleting WAL directory at /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785978402713339-22781-0/minicluster-data/ts-2/wal/wals/b21983c1ae0448a4a8fbc51d9a652aa5
I20260806 01:12:20.913491 21389 ts_tablet_manager.cc:1939] T b21983c1ae0448a4a8fbc51d9a652aa5 P 53d7d36fb01e46738e3cd140334dcff7: Deleting consensus metadata
I20260806 01:12:20.915249 20639 catalog_manager.cc:4841] TS 53d7d36fb01e46738e3cd140334dcff7 (127.22.63.67:35177): tablet b21983c1ae0448a4a8fbc51d9a652aa5 (table test-table [id=6dec572705194abbb9b4397807a6390f]) successfully deleted
I20260806 01:12:21.025369 21388 raft_consensus.cc:2258] T b21983c1ae0448a4a8fbc51d9a652aa5 P 1bdf219b25874cc5936365a0b6972430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 01:12:21.025729 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 20741
W20260806 01:12:21.052400 20641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.63.65:43247 (error 108)
I20260806 01:12:21.052693 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 20939
I20260806 01:12:21.055056 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.053923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60162)
0806 01:12:21.054771 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.054898 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":632,"threads_started":1}
W20260806 01:12:21.055310 20641 catalog_manager.cc:4584] TS 1bdf219b25874cc5936365a0b6972430 (127.22.63.65:43247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b21983c1ae0448a4a8fbc51d9a652aa5: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
I20260806 01:12:21.059731 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.053614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.22.63.66:37545)
0806 01:12:21.053813 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:21.053819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:21.053852 (+ 33us) server_negotiation.cc:407] Connection header received
0806 01:12:21.053987 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:21.053993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:21.054131 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:21.054278 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:21.054652 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:21.055470 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:21.056803 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:21.057251 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:21.057418 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:21.058353 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:21.058362 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:21.058375 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:21.058991 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:21.059037 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:21.059058 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:21.059144 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:21.059279 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 01:12:21.059365 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 01:12:21.060475 21114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:21.108811 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.108377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60164)
0806 01:12:21.108581 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.108666 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:12:21.109884 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.109507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60174)
0806 01:12:21.109676 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.109742 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 01:12:21.150626 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 20939
I20260806 01:12:21.170444 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.169971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60176)
0806 01:12:21.170195 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.170284 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:12:21.171535 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.171170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60188)
0806 01:12:21.171323 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.171392 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 01:12:21.176960 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21126
I20260806 01:12:21.183997 21352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.177783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.22.63.67:35177)
0806 01:12:21.177956 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:21.177962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:21.177982 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:12:21.178136 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:21.178140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:21.178243 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:21.178331 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:21.178765 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:21.179530 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:21.180932 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:21.181365 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:21.181492 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:21.182531 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:21.182540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:21.182557 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:21.183164 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:21.183220 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:21.183241 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:21.183336 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:21.183492 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 01:12:21.183583 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:12:21.184670 21314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:21.252367 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.251779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60190)
0806 01:12:21.252062 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.252187 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 01:12:21.253545 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:21.253117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.65:43247 (local address 127.0.0.1:60196)
0806 01:12:21.253327 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 01:12:21.253413 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.65:43247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 01:12:21.273952 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 21126
I20260806 01:12:21.299966 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 20621
I20260806 01:12:21.301147 20701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:21.414521 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 20621
2026-08-06T01:12:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3115 ms)
[----------] 6 tests from EncryptionPolicyTest (24785 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:21 dist-test-slave-7f01 krb5kdc[21407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:21 dist-test-slave-7f01 krb5kdc[21407](info): set up 2 sockets
Aug 06 01:12:21 dist-test-slave-7f01 krb5kdc[21407](info): commencing operation
krb5kdc: starting...
W20260806 01:12:22.767256 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:22 dist-test-slave-7f01 krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:23.095683 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:23.448557 21428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:23.448907 21428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:23.449082 21428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:23.456377 21428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:23.456555 21428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:23.456656 21428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:23.456758 21428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:23.456835 21428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:23.466866 21428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:23.468362 21428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:23.469763 21428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:23.481863 21436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:23.481909 21435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:23.481827 21441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:23.482600 21428 server_base.cc:1034] running on GCE node
I20260806 01:12:23.483490 21428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:23.485136 21428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:23.486373 21428 hybrid_clock.cc:648] HybridClock initialized: now 1785978743486311 us; error 38 us; skew 500 ppm
I20260806 01:12:23.486802 21428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:23 dist-test-slave-7f01 krb5kdc[21407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978743, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:23.490375 21428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:23.492038 21428 webserver.cc:460] Webserver started at http://127.0.0.1:44407/ using document root <none> and password file <none>
I20260806 01:12:23.492770 21428 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:23.492937 21428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:23.493304 21428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:23.496693 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "9ec4ceb1d2724229b067cd57e6228625"
format_stamp: "Formatted at 2026-08-06 01:12:23 on dist-test-slave-7f01"
server_key: "a19e73bc7864c6c08b0322f5a3942b6c"
server_key_iv: "2aba54e19e7e9d27e3c514b317f7b395"
server_key_version: "encryptionkey@0"
I20260806 01:12:23.497568 21428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "9ec4ceb1d2724229b067cd57e6228625"
format_stamp: "Formatted at 2026-08-06 01:12:23 on dist-test-slave-7f01"
server_key: "a19e73bc7864c6c08b0322f5a3942b6c"
server_key_iv: "2aba54e19e7e9d27e3c514b317f7b395"
server_key_version: "encryptionkey@0"
I20260806 01:12:23.503357 21428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 01:12:23.507197 21451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:23.508710 21428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:12:23.508931 21428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "9ec4ceb1d2724229b067cd57e6228625"
format_stamp: "Formatted at 2026-08-06 01:12:23 on dist-test-slave-7f01"
server_key: "a19e73bc7864c6c08b0322f5a3942b6c"
server_key_iv: "2aba54e19e7e9d27e3c514b317f7b395"
server_key_version: "encryptionkey@0"
I20260806 01:12:23.509136 21428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:23.536015 21428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:23.541105 21428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:23.541451 21428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:23.562682 21428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37475
I20260806 01:12:23.562719 21526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37475 every 8 connection(s)
I20260806 01:12:23.564282 21428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:23.568221 21527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:23.569339 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 21428
I20260806 01:12:23.569584 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:23.570327 22781 external_mini_cluster.cc:1428] Setting key 8bb45996524eeceaa12908df89be0146
I20260806 01:12:23.582847 21527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: Bootstrap starting.
Aug 06 01:12:23 dist-test-slave-7f01 krb5kdc[21407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:12:23.586309 21527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:23.587638 21527 log.cc:824] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:23.590852 21527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: No bootstrap required, opened a new log
I20260806 01:12:23.594517 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:23.572633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56665 (local address 127.0.0.1:37475)
0806 01:12:23.573492 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:23.573513 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:23.573565 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:12:23.575283 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:23.575319 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:23.575882 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:23.576323 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:23.577819 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:23.579930 (+ 2111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:23.581235 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:23.581908 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:23.582108 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:23.586493 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:23.586510 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:23.586528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:23.586566 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:23.590103 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:23.590931 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:23.590938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:23.590943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:23.591159 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:23.591656 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:23.591661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:23.591664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:23.592246 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:23.592596 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:23.593155 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 01:12:23.593489 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":353,"threads_started":1}
I20260806 01:12:23.598959 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:23.600617 21527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260806 01:12:23.601032 21527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:23.601154 21527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ec4ceb1d2724229b067cd57e6228625, State: Initialized, Role: FOLLOWER
I20260806 01:12:23.601732 21527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260806 01:12:23.602000 21527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:23.602193 21527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:23.602394 21527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:23.605540 21527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260806 01:12:23.606133 21527 leader_election.cc:302] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ec4ceb1d2724229b067cd57e6228625; no voters:
I20260806 01:12:23.606833 21527 leader_election.cc:288] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:23.606962 21539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:23.608537 21539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [term 1 LEADER]: Becoming Leader. State: Replica: 9ec4ceb1d2724229b067cd57e6228625, State: Running, Role: LEADER
I20260806 01:12:23.609082 21539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260806 01:12:23.609462 21527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:23.621984 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ec4ceb1d2724229b067cd57e6228625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } } }
I20260806 01:12:23.622117 21543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ec4ceb1d2724229b067cd57e6228625. Latest consensus state: current_term: 1 leader_uuid: "9ec4ceb1d2724229b067cd57e6228625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ec4ceb1d2724229b067cd57e6228625" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } } }
I20260806 01:12:23.622479 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:23.622493 21543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:23.624553 21531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:23.604327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36521 (local address 127.0.0.1:37475)
0806 01:12:23.604553 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:23.604559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:23.604851 (+ 292us) server_negotiation.cc:407] Connection header received
0806 01:12:23.605000 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:23.605006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:23.605089 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:23.605205 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:23.606870 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:23.608405 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:23.610415 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:23.611409 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:23.612507 (+ 1098us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:23.616061 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:23.616075 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:23.616083 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:23.616142 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:23.620609 (+ 4467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:23.621931 (+ 1322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:23.621939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:23.621945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:23.622158 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:23.622880 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:23.622884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:23.622886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:23.623207 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:23.623517 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:23.623986 (+ 469us) server_negotiation.cc:299] Negotiation successful
0806 01:12:23.624198 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 01:12:23.630494 21562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:12:23.630647 21562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:12:23.630852 21563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:23.633388 21563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:23.643333 21563 catalog_manager.cc:1347] Generated new cluster ID: d105795684414006858a794a8f3e9032
I20260806 01:12:23.643476 21563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:23.657626 21563 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:23.659590 21563 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:23.671763 21563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ec4ceb1d2724229b067cd57e6228625: Generated new TSK 0
I20260806 01:12:23.672653 21563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:23.679000 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21428
I20260806 01:12:23.681159 21520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:23.771708 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 21428
W20260806 01:12:23.798425 21405 connection.cc:535] client connection to 127.0.0.1:37475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37475): Connection reset by peer (error 104)
2026-08-06T01:12:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:23 dist-test-slave-7f01 krb5kdc[21569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:23 dist-test-slave-7f01 krb5kdc[21569](info): set up 2 sockets
Aug 06 01:12:23 dist-test-slave-7f01 krb5kdc[21569](info): commencing operation
krb5kdc: starting...
W20260806 01:12:25.116065 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:25 dist-test-slave-7f01 krb5kdc[21569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:25.464951 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:25.815315 21591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:25.815616 21591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:25.815842 21591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:25.823127 21591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:25.823241 21591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:25.823344 21591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:25.823457 21591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:25.823508 21591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:25.833642 21591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:25.835012 21591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:25.836440 21591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:25.848203 21598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:25.848290 21599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:25.848263 21601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:25.849416 21591 server_base.cc:1034] running on GCE node
I20260806 01:12:25.850246 21591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:25.851706 21591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:25.852975 21591 hybrid_clock.cc:648] HybridClock initialized: now 1785978745852919 us; error 48 us; skew 500 ppm
I20260806 01:12:25.853313 21591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260806 01:12:26.724860 21591 net_util.cc:504] Time spent look up canonical hostname for localhost 'dist-test-slave-7f01': real 0.871s user 0.000s sys 0.001s
Aug 06 01:12:26 dist-test-slave-7f01 krb5kdc[21569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978746, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:26.727438 21591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:26.729323 21591 webserver.cc:460] Webserver started at http://127.0.0.1:45147/ using document root <none> and password file <none>
I20260806 01:12:26.730036 21591 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:26.730134 21591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:26.730440 21591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:26.733651 21591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "885d900a67b74451a3bb289e0bc54d7e"
format_stamp: "Formatted at 2026-08-06 01:12:26 on dist-test-slave-7f01"
server_key: "6a1e05415e5e03600da9926b5d973a90"
server_key_iv: "237aa79620f0331b0c8f78dfcb2835b6"
server_key_version: "encryptionkey@0"
I20260806 01:12:26.734441 21591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "885d900a67b74451a3bb289e0bc54d7e"
format_stamp: "Formatted at 2026-08-06 01:12:26 on dist-test-slave-7f01"
server_key: "6a1e05415e5e03600da9926b5d973a90"
server_key_iv: "237aa79620f0331b0c8f78dfcb2835b6"
server_key_version: "encryptionkey@0"
I20260806 01:12:26.739866 21591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:12:26.743397 21610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:26.744846 21591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:12:26.745057 21591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "885d900a67b74451a3bb289e0bc54d7e"
format_stamp: "Formatted at 2026-08-06 01:12:26 on dist-test-slave-7f01"
server_key: "6a1e05415e5e03600da9926b5d973a90"
server_key_iv: "237aa79620f0331b0c8f78dfcb2835b6"
server_key_version: "encryptionkey@0"
I20260806 01:12:26.745292 21591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:26.792938 21591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:26.797567 21591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:26.797880 21591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:26.817410 21591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38131
I20260806 01:12:26.817385 21681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38131 every 8 connection(s)
I20260806 01:12:26.818842 21591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:26.822626 21682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:26.825544 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 21591
I20260806 01:12:26.825747 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:26.826198 22781 external_mini_cluster.cc:1428] Setting key 40342f6b7474294a2783b84177bd10ba
I20260806 01:12:26.834607 21682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e: Bootstrap starting.
I20260806 01:12:26.838470 21682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:26.839622 21682 log.cc:824] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:26.839991 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:26.828127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42949 (local address 127.0.0.1:38131)
0806 01:12:26.828887 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:26.828898 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:26.828942 (+ 44us) server_negotiation.cc:407] Connection header received
0806 01:12:26.830625 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:26.830647 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:26.831058 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:26.831375 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:26.832043 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:26.833374 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:26.835290 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:26.836030 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:26.836184 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:26.837574 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:26.837582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:26.837595 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:26.838131 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:26.838203 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:26.838227 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:26.838359 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:26.838746 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 01:12:26.838939 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":341,"threads_started":1}
I20260806 01:12:26.842909 21682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e: No bootstrap required, opened a new log
I20260806 01:12:26.844163 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:26.853681 21682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260806 01:12:26.854187 21682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:26.854321 21682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 885d900a67b74451a3bb289e0bc54d7e, State: Initialized, Role: FOLLOWER
I20260806 01:12:26.855010 21682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260806 01:12:26.855268 21682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:26.855438 21682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:26.855623 21682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:26.857017 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:26.849757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47687 (local address 127.0.0.1:38131)
0806 01:12:26.849974 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:26.849980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:26.850420 (+ 440us) server_negotiation.cc:407] Connection header received
0806 01:12:26.850586 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:26.850594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:26.850687 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:26.850792 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:26.851364 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:26.852615 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:26.854052 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:26.854694 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:26.854836 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:26.856194 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:26.856199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:26.856202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:26.856220 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:26.856261 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:26.856279 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:26.856424 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:26.856631 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 01:12:26.856693 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 01:12:26.858407 21682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260806 01:12:26.858981 21682 leader_election.cc:302] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 885d900a67b74451a3bb289e0bc54d7e; no voters:
I20260806 01:12:26.859531 21682 leader_election.cc:288] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:26.859714 21695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:26.860461 21695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [term 1 LEADER]: Becoming Leader. State: Replica: 885d900a67b74451a3bb289e0bc54d7e, State: Running, Role: LEADER
I20260806 01:12:26.861080 21695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } }
I20260806 01:12:26.861987 21682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:26.864259 21696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "885d900a67b74451a3bb289e0bc54d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260806 01:12:26.864684 21696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:26.864382 21699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 885d900a67b74451a3bb289e0bc54d7e. Latest consensus state: current_term: 1 leader_uuid: "885d900a67b74451a3bb289e0bc54d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "885d900a67b74451a3bb289e0bc54d7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38131 } } }
I20260806 01:12:26.865937 21699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:26.868414 21703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:26.874465 21703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:26.883913 21703 catalog_manager.cc:1347] Generated new cluster ID: 3cc9f9d4893c4e9bbf7c7fdfea7427e2
I20260806 01:12:26.884063 21703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:26.895251 21703 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:26.897517 21703 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:26.904604 21703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 885d900a67b74451a3bb289e0bc54d7e: Generated new TSK 0
I20260806 01:12:26.905603 21703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:26.924242 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21591
W20260806 01:12:26.926709 21676 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42949
W20260806 01:12:26.927594 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21591: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:26.927809 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 21591
W20260806 01:12:26.958089 21567 connection.cc:535] client connection to 127.0.0.1:38131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38131): Connection reset by peer (error 104)
2026-08-06T01:12:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (3159 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:27 dist-test-slave-7f01 krb5kdc[21727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:27 dist-test-slave-7f01 krb5kdc[21727](info): set up 2 sockets
Aug 06 01:12:27 dist-test-slave-7f01 krb5kdc[21727](info): commencing operation
krb5kdc: starting...
W20260806 01:12:28.284610 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:28 dist-test-slave-7f01 krb5kdc[21727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:28.627349 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:28.977504 21748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:28.977807 21748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:28.977983 21748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:28.985440 21748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:28.985559 21748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:28.985672 21748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:28.985781 21748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:28.985846 21748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:28.996049 21748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38349
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:28.997376 21748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:28.998730 21748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:29.010864 21754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:29.010823 21757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:29.010906 21755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:29.012055 21748 server_base.cc:1034] running on GCE node
I20260806 01:12:29.012836 21748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:29.014286 21748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:29.015522 21748 hybrid_clock.cc:648] HybridClock initialized: now 1785978749015476 us; error 46 us; skew 500 ppm
I20260806 01:12:29.015936 21748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:29 dist-test-slave-7f01 krb5kdc[21727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978749, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:29.019083 21748 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:29.020576 21748 webserver.cc:460] Webserver started at http://127.0.0.1:40913/ using document root <none> and password file <none>
I20260806 01:12:29.021207 21748 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:29.021308 21748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:29.021577 21748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:29.024552 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "ef751209ac9f46c6a3a07f969500b6e9"
format_stamp: "Formatted at 2026-08-06 01:12:29 on dist-test-slave-7f01"
server_key: "16a2f512d562113ed1acae5933d9b05c"
server_key_iv: "eca732a77c84e321369cc1d7b99fcc2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:29.025277 21748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "ef751209ac9f46c6a3a07f969500b6e9"
format_stamp: "Formatted at 2026-08-06 01:12:29 on dist-test-slave-7f01"
server_key: "16a2f512d562113ed1acae5933d9b05c"
server_key_iv: "eca732a77c84e321369cc1d7b99fcc2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:29.030321 21748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 01:12:29.033838 21770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:29.035157 21748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:29.035298 21748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "ef751209ac9f46c6a3a07f969500b6e9"
format_stamp: "Formatted at 2026-08-06 01:12:29 on dist-test-slave-7f01"
server_key: "16a2f512d562113ed1acae5933d9b05c"
server_key_iv: "eca732a77c84e321369cc1d7b99fcc2a"
server_key_version: "encryptionkey@0"
I20260806 01:12:29.035470 21748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:29.041236 21748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:29.045244 21748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:29.045506 21748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:29.064584 21748 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38349
I20260806 01:12:29.064635 21846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38349 every 8 connection(s)
I20260806 01:12:29.066072 21748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:29.069944 21847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:29.071947 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 21748
I20260806 01:12:29.072145 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:29.072711 22781 external_mini_cluster.cc:1428] Setting key 3c88df38ff483b14fb86847319f39a76
I20260806 01:12:29.080556 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:29.074506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39307 (local address 127.0.0.1:38349)
0806 01:12:29.075358 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:29.075369 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:29.075411 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:12:29.077039 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:29.077062 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:29.077447 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:29.077768 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:29.078898 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:29.078926 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:29.078939 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:29.078953 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:29.079017 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:29.079037 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:29.079218 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:29.079364 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 01:12:29.079567 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":415,"threads_started":1}
I20260806 01:12:29.084575 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:29.084551 21847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9: Bootstrap starting.
I20260806 01:12:29.088519 21847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:29.089883 21847 log.cc:824] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:29.092635 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:29.090048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46689 (local address 127.0.0.1:38349)
0806 01:12:29.090211 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:29.090217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:29.090631 (+ 414us) server_negotiation.cc:407] Connection header received
0806 01:12:29.090836 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:29.090840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:29.090888 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:29.091016 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:29.092011 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:29.092020 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:29.092023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:29.092037 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:29.092074 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:29.092093 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:29.092211 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:29.092314 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 01:12:29.092374 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 01:12:29.093142 21847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9: No bootstrap required, opened a new log
I20260806 01:12:29.101446 21847 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } }
I20260806 01:12:29.101778 21847 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:29.101840 21847 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef751209ac9f46c6a3a07f969500b6e9, State: Initialized, Role: FOLLOWER
I20260806 01:12:29.102332 21847 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } }
I20260806 01:12:29.102581 21847 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:29.102732 21847 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:29.102907 21847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:29.105108 21847 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } }
I20260806 01:12:29.105592 21847 leader_election.cc:302] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef751209ac9f46c6a3a07f969500b6e9; no voters:
I20260806 01:12:29.106171 21847 leader_election.cc:288] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:29.106305 21859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:29.107002 21859 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [term 1 LEADER]: Becoming Leader. State: Replica: ef751209ac9f46c6a3a07f969500b6e9, State: Running, Role: LEADER
I20260806 01:12:29.107545 21859 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } }
I20260806 01:12:29.108554 21847 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:29.110347 21861 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef751209ac9f46c6a3a07f969500b6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } } }
I20260806 01:12:29.110674 21861 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:29.112278 21871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:29.112372 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef751209ac9f46c6a3a07f969500b6e9. Latest consensus state: current_term: 1 leader_uuid: "ef751209ac9f46c6a3a07f969500b6e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef751209ac9f46c6a3a07f969500b6e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38349 } } }
I20260806 01:12:29.112646 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:29.117554 21871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:29.125798 21871 catalog_manager.cc:1347] Generated new cluster ID: fde16ef3a2ec4abfbea901a0914177ee
I20260806 01:12:29.125926 21871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:29.161195 21871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:29.161718 21871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:29.168423 21871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef751209ac9f46c6a3a07f969500b6e9: Generated new TSK 0
I20260806 01:12:29.169399 21871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:29.182929 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21748
W20260806 01:12:29.184926 21839 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39307
W20260806 01:12:29.185621 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:29.185774 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 21748
2026-08-06T01:12:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:29 dist-test-slave-7f01 krb5kdc[21897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:29 dist-test-slave-7f01 krb5kdc[21897](info): set up 2 sockets
Aug 06 01:12:29 dist-test-slave-7f01 krb5kdc[21897](info): commencing operation
krb5kdc: starting...
W20260806 01:12:30.519570 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:30 dist-test-slave-7f01 krb5kdc[21897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:30.870357 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43769
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:31.214426 21922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:31.214762 21922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:31.214934 21922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:31.221935 21922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:31.222086 21922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:31.222198 21922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:31.222309 21922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:31.222396 21922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:31.231992 21922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:31.233287 21922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:31.234611 21922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:31.246151 21933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:31.246169 21934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:31.246110 21937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:31.247053 21922 server_base.cc:1034] running on GCE node
I20260806 01:12:31.247962 21922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:31.249356 21922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:31.250568 21922 hybrid_clock.cc:648] HybridClock initialized: now 1785978751250524 us; error 41 us; skew 500 ppm
I20260806 01:12:31.250914 21922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:31 dist-test-slave-7f01 krb5kdc[21897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978751, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:31.254285 21922 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:31.255939 21922 webserver.cc:460] Webserver started at http://127.0.0.1:35051/ using document root <none> and password file <none>
I20260806 01:12:31.256634 21922 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:31.256768 21922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:31.257094 21922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:31.260164 21922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "0370ca62f4334176bc1d36dabec3e632"
format_stamp: "Formatted at 2026-08-06 01:12:31 on dist-test-slave-7f01"
server_key: "3510de26f1bb2e9c222e6cd8c831e568"
server_key_iv: "800d911700da8c45ba76bb87f10b6489"
server_key_version: "encryptionkey@0"
I20260806 01:12:31.260958 21922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "0370ca62f4334176bc1d36dabec3e632"
format_stamp: "Formatted at 2026-08-06 01:12:31 on dist-test-slave-7f01"
server_key: "3510de26f1bb2e9c222e6cd8c831e568"
server_key_iv: "800d911700da8c45ba76bb87f10b6489"
server_key_version: "encryptionkey@0"
I20260806 01:12:31.266211 21922 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:12:31.269719 21945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:31.271130 21922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:31.271319 21922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "0370ca62f4334176bc1d36dabec3e632"
format_stamp: "Formatted at 2026-08-06 01:12:31 on dist-test-slave-7f01"
server_key: "3510de26f1bb2e9c222e6cd8c831e568"
server_key_iv: "800d911700da8c45ba76bb87f10b6489"
server_key_version: "encryptionkey@0"
I20260806 01:12:31.271528 21922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:31.298347 21922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:31.302893 21922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:31.303210 21922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:31.322562 21922 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43769
I20260806 01:12:31.322597 22025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43769 every 8 connection(s)
I20260806 01:12:31.324091 21922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:31.327942 22026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:31.334357 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 21922
I20260806 01:12:31.334553 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:31.335180 22781 external_mini_cluster.cc:1428] Setting key 1f3af40cdb9104b6080446f2e21bcf42
I20260806 01:12:31.340332 22026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632: Bootstrap starting.
I20260806 01:12:31.344473 22026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:31.345845 22026 log.cc:824] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:31.348978 22026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632: No bootstrap required, opened a new log
Aug 06 01:12:31 dist-test-slave-7f01 krb5kdc[21897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:12:31.357532 22026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } }
I20260806 01:12:31.357935 22026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:31.358050 22026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0370ca62f4334176bc1d36dabec3e632, State: Initialized, Role: FOLLOWER
I20260806 01:12:31.358740 22026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } }
I20260806 01:12:31.359056 22026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:31.359212 22026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:31.359407 22026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:31.361029 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:31.337564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56893 (local address 127.0.0.1:43769)
0806 01:12:31.338842 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:31.338860 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:31.338913 (+ 53us) server_negotiation.cc:407] Connection header received
0806 01:12:31.341364 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:31.341393 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:31.341907 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:31.342412 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:31.343815 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:31.345894 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:31.347237 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:31.347977 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:31.348184 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:31.352685 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:31.352693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:31.352711 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:31.352759 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:31.356075 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:31.356840 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:31.356848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:31.356855 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:31.357029 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:31.357432 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:31.357438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:31.357441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:31.358120 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:31.358396 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:31.359053 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 01:12:31.359572 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":775,"threads_started":1}
I20260806 01:12:31.362458 22026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } }
I20260806 01:12:31.363008 22026 leader_election.cc:302] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0370ca62f4334176bc1d36dabec3e632; no voters:
I20260806 01:12:31.363680 22026 leader_election.cc:288] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:31.363791 22035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:31.365046 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:31.365345 22035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [term 1 LEADER]: Becoming Leader. State: Replica: 0370ca62f4334176bc1d36dabec3e632, State: Running, Role: LEADER
I20260806 01:12:31.365865 22035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } }
I20260806 01:12:31.366235 22026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:31.368368 22039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0370ca62f4334176bc1d36dabec3e632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } } }
I20260806 01:12:31.368753 22039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:31.369313 22037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0370ca62f4334176bc1d36dabec3e632. Latest consensus state: current_term: 1 leader_uuid: "0370ca62f4334176bc1d36dabec3e632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0370ca62f4334176bc1d36dabec3e632" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43769 } } }
I20260806 01:12:31.369684 22037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:31.375844 22044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:31.383566 22044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:31.399525 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:31.378682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45381 (local address 127.0.0.1:43769)
0806 01:12:31.378874 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:31.378881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:31.380056 (+ 1175us) server_negotiation.cc:407] Connection header received
0806 01:12:31.380341 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:31.380347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:31.380432 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:31.380509 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:31.382824 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:31.384001 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:31.385852 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:31.386519 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:31.386632 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:31.390227 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:31.390235 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:31.390239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:31.390280 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:31.394869 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:31.396089 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:31.396097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:31.396103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:31.396293 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:31.396896 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:31.396904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:31.396911 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:31.397475 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:31.397775 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:31.398535 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 01:12:31.398846 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:12:31.405507 22044 catalog_manager.cc:1347] Generated new cluster ID: d5cb952a808446e68fe16043648d7a95
I20260806 01:12:31.405620 22044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:31.417294 22044 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:31.419198 22044 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:31.427045 22044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0370ca62f4334176bc1d36dabec3e632: Generated new TSK 0
I20260806 01:12:31.427928 22044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:31.441128 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 21922
I20260806 01:12:31.442332 22018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:31.533067 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 21922
2026-08-06T01:12:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:31 dist-test-slave-7f01 krb5kdc[22073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:31 dist-test-slave-7f01 krb5kdc[22073](info): set up 2 sockets
Aug 06 01:12:31 dist-test-slave-7f01 krb5kdc[22073](info): commencing operation
krb5kdc: starting...
W20260806 01:12:32.886365 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:33 dist-test-slave-7f01 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:33.236095 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:33.586742 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:33.587065 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:33.587229 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:33.594416 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:33.594534 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:33.594650 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:33.594754 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:33.594807 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:33.604578 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:33.605928 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:33.607278 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:33.619396 22106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:33.619451 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:33.619451 22107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:33.620189 22097 server_base.cc:1034] running on GCE node
I20260806 01:12:33.620993 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:33.622361 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:33.623600 22097 hybrid_clock.cc:648] HybridClock initialized: now 1785978753623557 us; error 53 us; skew 500 ppm
I20260806 01:12:33.624017 22097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:33 dist-test-slave-7f01 krb5kdc[22073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:33.627280 22097 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:33.628785 22097 webserver.cc:460] Webserver started at http://127.0.0.1:41633/ using document root <none> and password file <none>
I20260806 01:12:33.629514 22097 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:33.629665 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:33.629972 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:33.633010 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "debf9d7200f74bb49122cbb0727ac54f"
format_stamp: "Formatted at 2026-08-06 01:12:33 on dist-test-slave-7f01"
server_key: "a8bb226ca356159e6dbc2ec2e34fcec0"
server_key_iv: "f5b90a095c9804b24f8e0e058cabd7be"
server_key_version: "encryptionkey@0"
I20260806 01:12:33.633780 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "debf9d7200f74bb49122cbb0727ac54f"
format_stamp: "Formatted at 2026-08-06 01:12:33 on dist-test-slave-7f01"
server_key: "a8bb226ca356159e6dbc2ec2e34fcec0"
server_key_iv: "f5b90a095c9804b24f8e0e058cabd7be"
server_key_version: "encryptionkey@0"
I20260806 01:12:33.639153 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:12:33.642773 22118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:33.644235 22097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:33.644433 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "debf9d7200f74bb49122cbb0727ac54f"
format_stamp: "Formatted at 2026-08-06 01:12:33 on dist-test-slave-7f01"
server_key: "a8bb226ca356159e6dbc2ec2e34fcec0"
server_key_iv: "f5b90a095c9804b24f8e0e058cabd7be"
server_key_version: "encryptionkey@0"
I20260806 01:12:33.644636 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:33.674530 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:33.678892 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:33.679236 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:33.698293 22097 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36595
I20260806 01:12:33.698336 22190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36595 every 8 connection(s)
I20260806 01:12:33.699692 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:33.700484 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22097
I20260806 01:12:33.700701 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:33.701274 22781 external_mini_cluster.cc:1428] Setting key 82910846897c3fb4479604e8c965e4ea
I20260806 01:12:33.704259 22191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:33.714082 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:33.703176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35361 (local address 127.0.0.1:36595)
0806 01:12:33.703996 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:33.704008 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:33.704050 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:12:33.705675 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:33.705705 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:33.706083 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:33.706454 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:33.707051 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:33.708433 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:33.709756 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:33.710414 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:33.710617 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:33.711869 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:33.711878 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:33.711891 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:33.712439 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:33.712515 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:33.712539 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:33.712687 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:33.712929 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 01:12:33.713177 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":335,"threads_started":1}
I20260806 01:12:33.714978 22191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f: Bootstrap starting.
I20260806 01:12:33.717715 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:33.718389 22191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:33.719360 22191 log.cc:824] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:33.722748 22191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f: No bootstrap required, opened a new log
I20260806 01:12:33.728564 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:33.722265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55013 (local address 127.0.0.1:36595)
0806 01:12:33.722416 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:33.722420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:33.722988 (+ 568us) server_negotiation.cc:407] Connection header received
0806 01:12:33.723213 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:33.723217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:33.723278 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:33.723392 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:33.723928 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:33.724658 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:33.726002 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:33.726469 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:33.726618 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:33.727825 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:33.727838 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:33.727842 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:33.727859 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:33.727900 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:33.727916 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:33.728026 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:33.728176 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 01:12:33.728274 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 01:12:33.730854 22191 raft_consensus.cc:348] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } }
I20260806 01:12:33.731236 22191 raft_consensus.cc:374] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:33.731360 22191 raft_consensus.cc:729] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: debf9d7200f74bb49122cbb0727ac54f, State: Initialized, Role: FOLLOWER
I20260806 01:12:33.731963 22191 consensus_queue.cc:260] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } }
I20260806 01:12:33.732203 22191 raft_consensus.cc:388] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:33.732362 22191 raft_consensus.cc:482] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:33.732540 22191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:33.734987 22191 raft_consensus.cc:504] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } }
I20260806 01:12:33.735528 22191 leader_election.cc:302] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: debf9d7200f74bb49122cbb0727ac54f; no voters:
I20260806 01:12:33.736122 22191 leader_election.cc:288] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:33.736307 22205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:33.736991 22205 raft_consensus.cc:686] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [term 1 LEADER]: Becoming Leader. State: Replica: debf9d7200f74bb49122cbb0727ac54f, State: Running, Role: LEADER
I20260806 01:12:33.737530 22205 consensus_queue.cc:237] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } }
I20260806 01:12:33.738214 22191 sys_catalog.cc:564] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:33.740468 22208 sys_catalog.cc:455] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [sys.catalog]: SysCatalogTable state changed. Reason: New leader debf9d7200f74bb49122cbb0727ac54f. Latest consensus state: current_term: 1 leader_uuid: "debf9d7200f74bb49122cbb0727ac54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } } }
I20260806 01:12:33.740859 22208 sys_catalog.cc:458] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:33.740588 22206 sys_catalog.cc:455] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "debf9d7200f74bb49122cbb0727ac54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "debf9d7200f74bb49122cbb0727ac54f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36595 } } }
I20260806 01:12:33.740979 22206 sys_catalog.cc:458] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:33.741791 22216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:33.749445 22216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:33.758136 22216 catalog_manager.cc:1347] Generated new cluster ID: beba25c8a1654bc293379df2fc91cf9a
I20260806 01:12:33.758251 22216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:33.779587 22216 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:33.781066 22216 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:33.795181 22216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P debf9d7200f74bb49122cbb0727ac54f: Generated new TSK 0
I20260806 01:12:33.796113 22216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:33.811719 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22097
W20260806 01:12:33.813185 22186 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35361
W20260806 01:12:33.813966 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22097: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:33.814103 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22097
2026-08-06T01:12:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:33 dist-test-slave-7f01 krb5kdc[22234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:33 dist-test-slave-7f01 krb5kdc[22234](info): set up 2 sockets
Aug 06 01:12:33 dist-test-slave-7f01 krb5kdc[22234](info): commencing operation
krb5kdc: starting...
W20260806 01:12:35.172029 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:35 dist-test-slave-7f01 krb5kdc[22234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:35.521359 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:35.870548 22253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:35.870853 22253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:35.871030 22253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:35.880540 22253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:35.880702 22253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:35.880795 22253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:35.880903 22253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:35.880977 22253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:35.891615 22253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:35.892930 22253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:35.894281 22253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:35.906245 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:35.906201 22265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:35.906263 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:35.907168 22253 server_base.cc:1034] running on GCE node
I20260806 01:12:35.907965 22253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:35.909255 22253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:35.910470 22253 hybrid_clock.cc:648] HybridClock initialized: now 1785978755910426 us; error 30 us; skew 500 ppm
I20260806 01:12:35.910774 22253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:35 dist-test-slave-7f01 krb5kdc[22234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:35.913937 22253 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:35.915274 22253 webserver.cc:460] Webserver started at http://127.0.0.1:43517/ using document root <none> and password file <none>
I20260806 01:12:35.915982 22253 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:35.916091 22253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:35.916350 22253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:35.919183 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "eb47129416b4477f85f02e92c212b5c3"
format_stamp: "Formatted at 2026-08-06 01:12:35 on dist-test-slave-7f01"
server_key: "6cb14dfe2525090c8918b3a4a6160046"
server_key_iv: "8a5a3842d3b9ae3e3a8f873e31061c49"
server_key_version: "encryptionkey@0"
I20260806 01:12:35.919950 22253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "eb47129416b4477f85f02e92c212b5c3"
format_stamp: "Formatted at 2026-08-06 01:12:35 on dist-test-slave-7f01"
server_key: "6cb14dfe2525090c8918b3a4a6160046"
server_key_iv: "8a5a3842d3b9ae3e3a8f873e31061c49"
server_key_version: "encryptionkey@0"
I20260806 01:12:35.924909 22253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260806 01:12:35.928287 22275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:35.929535 22253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:12:35.929674 22253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "eb47129416b4477f85f02e92c212b5c3"
format_stamp: "Formatted at 2026-08-06 01:12:35 on dist-test-slave-7f01"
server_key: "6cb14dfe2525090c8918b3a4a6160046"
server_key_iv: "8a5a3842d3b9ae3e3a8f873e31061c49"
server_key_version: "encryptionkey@0"
I20260806 01:12:35.929833 22253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:35.935350 22253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:35.939138 22253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:35.939388 22253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:35.957513 22253 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37287
I20260806 01:12:35.957553 22351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37287 every 8 connection(s)
I20260806 01:12:35.958865 22253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:35.962543 22352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:35.964627 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22253
I20260806 01:12:35.964807 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:35.965267 22781 external_mini_cluster.cc:1428] Setting key 469b67d40f0f2326a332998e8c3c2a6c
I20260806 01:12:35.972884 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:35.966988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51405 (local address 127.0.0.1:37287)
0806 01:12:35.967834 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:35.967846 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:35.967892 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:12:35.969507 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:35.969528 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:35.969824 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:35.970129 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:35.971248 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:35.971258 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:35.971270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:35.971284 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:35.971347 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:35.971370 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:35.971542 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:35.971696 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:12:35.971976 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":372,"threads_started":1}
I20260806 01:12:35.974820 22352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3: Bootstrap starting.
I20260806 01:12:35.976852 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:35.978636 22352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:35.979925 22352 log.cc:824] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:35.982877 22352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3: No bootstrap required, opened a new log
I20260806 01:12:35.984777 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:35.982367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38293 (local address 127.0.0.1:37287)
0806 01:12:35.982549 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:35.982554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:35.982873 (+ 319us) server_negotiation.cc:407] Connection header received
0806 01:12:35.982986 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:35.982991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:35.983045 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:35.983154 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:35.984125 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:35.984136 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:35.984141 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:35.984161 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:35.984204 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:35.984223 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:35.984340 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:35.984415 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 01:12:35.984479 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:12:35.990562 22352 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } }
I20260806 01:12:35.990914 22352 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:35.991037 22352 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb47129416b4477f85f02e92c212b5c3, State: Initialized, Role: FOLLOWER
I20260806 01:12:35.991582 22352 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } }
I20260806 01:12:35.991899 22352 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:35.992084 22352 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:35.992307 22352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:35.994426 22352 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } }
I20260806 01:12:35.994894 22352 leader_election.cc:302] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb47129416b4477f85f02e92c212b5c3; no voters:
I20260806 01:12:35.995479 22352 leader_election.cc:288] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:35.995656 22368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:35.996905 22368 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [term 1 LEADER]: Becoming Leader. State: Replica: eb47129416b4477f85f02e92c212b5c3, State: Running, Role: LEADER
I20260806 01:12:35.997399 22368 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } }
I20260806 01:12:35.997656 22352 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:36.000352 22369 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb47129416b4477f85f02e92c212b5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } } }
I20260806 01:12:36.000720 22369 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:36.001596 22380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:36.001910 22370 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb47129416b4477f85f02e92c212b5c3. Latest consensus state: current_term: 1 leader_uuid: "eb47129416b4477f85f02e92c212b5c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb47129416b4477f85f02e92c212b5c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37287 } } }
I20260806 01:12:36.002249 22370 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:36.007333 22380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:36.016654 22380 catalog_manager.cc:1347] Generated new cluster ID: b89e3fdc30e546a7bec993fd564eac37
I20260806 01:12:36.016790 22380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:36.026525 22380 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:36.026994 22380 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:36.041076 22380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb47129416b4477f85f02e92c212b5c3: Generated new TSK 0
I20260806 01:12:36.041870 22380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:36.054803 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22253
W20260806 01:12:36.055943 22347 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51405
W20260806 01:12:36.056605 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22253: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:36.056763 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22253
2026-08-06T01:12:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:36 dist-test-slave-7f01 krb5kdc[22399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:36 dist-test-slave-7f01 krb5kdc[22399](info): set up 2 sockets
Aug 06 01:12:36 dist-test-slave-7f01 krb5kdc[22399](info): commencing operation
krb5kdc: starting...
W20260806 01:12:37.400970 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:37 dist-test-slave-7f01 krb5kdc[22399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:37.743117 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:38.104197 22417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:38.104490 22417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:38.104651 22417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:38.111935 22417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:38.112053 22417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:38.112143 22417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:38.112231 22417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:38.112296 22417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:38.122216 22417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:38.123615 22417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:38.124974 22417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:38.137625 22428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:38.137663 22425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:38.137706 22426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:38.138485 22417 server_base.cc:1034] running on GCE node
I20260806 01:12:38.139312 22417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:38.140774 22417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:38.142019 22417 hybrid_clock.cc:648] HybridClock initialized: now 1785978758141956 us; error 47 us; skew 500 ppm
I20260806 01:12:38.142380 22417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:38 dist-test-slave-7f01 krb5kdc[22399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978758, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:38.148072 22417 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:38.149631 22417 webserver.cc:460] Webserver started at http://127.0.0.1:40827/ using document root <none> and password file <none>
I20260806 01:12:38.150270 22417 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:38.150382 22417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:38.150640 22417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:38.153656 22417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "3bb2f174f0044e639b8e5c6f51d7c635"
format_stamp: "Formatted at 2026-08-06 01:12:38 on dist-test-slave-7f01"
server_key: "f6e0ed651ad18572c70100477ff21ec1"
server_key_iv: "92ef896f4ef59e7577d8f6e856cdb72e"
server_key_version: "encryptionkey@0"
I20260806 01:12:38.154390 22417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "3bb2f174f0044e639b8e5c6f51d7c635"
format_stamp: "Formatted at 2026-08-06 01:12:38 on dist-test-slave-7f01"
server_key: "f6e0ed651ad18572c70100477ff21ec1"
server_key_iv: "92ef896f4ef59e7577d8f6e856cdb72e"
server_key_version: "encryptionkey@0"
I20260806 01:12:38.159693 22417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 01:12:38.163043 22437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:38.164397 22417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:38.164587 22417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "3bb2f174f0044e639b8e5c6f51d7c635"
format_stamp: "Formatted at 2026-08-06 01:12:38 on dist-test-slave-7f01"
server_key: "f6e0ed651ad18572c70100477ff21ec1"
server_key_iv: "92ef896f4ef59e7577d8f6e856cdb72e"
server_key_version: "encryptionkey@0"
I20260806 01:12:38.164762 22417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:38.191504 22417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:38.196139 22417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:38.196458 22417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:38.215672 22417 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39203
I20260806 01:12:38.215792 22515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39203 every 8 connection(s)
I20260806 01:12:38.217085 22417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:38.220515 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22417
I20260806 01:12:38.220713 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:38.221133 22516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:38.221274 22781 external_mini_cluster.cc:1428] Setting key dccac74f30fbaf58ed2b2a6d55d834eb
I20260806 01:12:38.236055 22516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635: Bootstrap starting.
Aug 06 01:12:38 dist-test-slave-7f01 krb5kdc[22399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:12:38.239403 22516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:38.240592 22516 log.cc:824] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:38.243408 22516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635: No bootstrap required, opened a new log
I20260806 01:12:38.244968 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:38.223206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:49791 (local address 127.0.0.1:39203)
0806 01:12:38.224037 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:38.224048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:38.224089 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:12:38.225916 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:38.225939 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:38.226327 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:38.226672 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:38.228617 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:38.229874 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:38.231723 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:38.232473 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:38.232621 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:38.237677 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:38.237685 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:38.237698 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:38.237734 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:38.240943 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:38.241741 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:38.241747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:38.241752 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:38.241916 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:38.242389 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:38.242394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:38.242396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:38.242872 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:38.243216 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:38.243696 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 01:12:38.244038 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":355,"threads_started":1}
I20260806 01:12:38.248816 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:38.251772 22516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } }
I20260806 01:12:38.252245 22516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:38.252370 22516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bb2f174f0044e639b8e5c6f51d7c635, State: Initialized, Role: FOLLOWER
I20260806 01:12:38.252929 22516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } }
I20260806 01:12:38.253274 22516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:38.253500 22516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:38.253674 22516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:38.256358 22516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } }
I20260806 01:12:38.256917 22516 leader_election.cc:302] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bb2f174f0044e639b8e5c6f51d7c635; no voters:
I20260806 01:12:38.257591 22516 leader_election.cc:288] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:38.257732 22530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:38.258394 22530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [term 1 LEADER]: Becoming Leader. State: Replica: 3bb2f174f0044e639b8e5c6f51d7c635, State: Running, Role: LEADER
I20260806 01:12:38.259060 22530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } }
I20260806 01:12:38.262002 22532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } } }
I20260806 01:12:38.262382 22532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:38.262410 22516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:38.262586 22531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bb2f174f0044e639b8e5c6f51d7c635. Latest consensus state: current_term: 1 leader_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bb2f174f0044e639b8e5c6f51d7c635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39203 } } }
I20260806 01:12:38.263638 22531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:38.264014 22535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:38.273245 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:38.253297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:55697 (local address 127.0.0.1:39203)
0806 01:12:38.253500 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:38.253506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:38.253953 (+ 447us) server_negotiation.cc:407] Connection header received
0806 01:12:38.254088 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:38.254096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:38.254177 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:38.254290 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:38.255686 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:38.256832 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:38.258737 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:38.259441 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:38.259547 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:38.262519 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:38.262527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:38.262534 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:38.262589 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:38.267460 (+ 4871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:38.268886 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:38.268893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:38.268900 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:38.269106 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:38.269879 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:38.269886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:38.269892 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:38.270426 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:38.272252 (+ 1826us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:38.272476 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 01:12:38.272724 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 01:12:38.279043 22535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:38.290334 22535 catalog_manager.cc:1347] Generated new cluster ID: 762c0a5a0a854629b6ba19d133e7269b
I20260806 01:12:38.290494 22535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:38.309850 22535 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:38.311172 22535 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:38.320348 22535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bb2f174f0044e639b8e5c6f51d7c635: Generated new TSK 0
I20260806 01:12:38.321219 22535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:38.330329 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22417
I20260806 01:12:38.331893 22511 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:38.424602 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22417
W20260806 01:12:38.448392 22396 connection.cc:535] client connection to 127.0.0.1:39203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39203): Connection reset by peer (error 104)
2026-08-06T01:12:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:38 dist-test-slave-7f01 krb5kdc[22559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:38 dist-test-slave-7f01 krb5kdc[22559](info): set up 2 sockets
Aug 06 01:12:38 dist-test-slave-7f01 krb5kdc[22559](info): commencing operation
krb5kdc: starting...
W20260806 01:12:39.763383 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:39 dist-test-slave-7f01 krb5kdc[22559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:40.103374 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:40.450465 22575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:40.450788 22575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:40.450968 22575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:40.458061 22575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:40.458207 22575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:40.458314 22575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:40.458431 22575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:40.458516 22575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:40.468246 22575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:40.469548 22575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:40.470846 22575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:40.482929 22589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:40.483016 22586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:40.482973 22581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:40.484543 22575 server_base.cc:1034] running on GCE node
I20260806 01:12:40.485592 22575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:40.487215 22575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:40.488466 22575 hybrid_clock.cc:648] HybridClock initialized: now 1785978760488428 us; error 53 us; skew 500 ppm
I20260806 01:12:40.488894 22575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:40 dist-test-slave-7f01 krb5kdc[22559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:40.492290 22575 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:40.493876 22575 webserver.cc:460] Webserver started at http://127.0.0.1:42977/ using document root <none> and password file <none>
I20260806 01:12:40.494575 22575 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:40.494725 22575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:40.495046 22575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:40.498155 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a"
format_stamp: "Formatted at 2026-08-06 01:12:40 on dist-test-slave-7f01"
server_key: "658285e13ce9ab883721cdeea080812c"
server_key_iv: "df4acc6936029912e57edb03c8e4576d"
server_key_version: "encryptionkey@0"
I20260806 01:12:40.498939 22575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a"
format_stamp: "Formatted at 2026-08-06 01:12:40 on dist-test-slave-7f01"
server_key: "658285e13ce9ab883721cdeea080812c"
server_key_iv: "df4acc6936029912e57edb03c8e4576d"
server_key_version: "encryptionkey@0"
I20260806 01:12:40.504083 22575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:12:40.507402 22597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:40.508702 22575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:40.508890 22575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a"
format_stamp: "Formatted at 2026-08-06 01:12:40 on dist-test-slave-7f01"
server_key: "658285e13ce9ab883721cdeea080812c"
server_key_iv: "df4acc6936029912e57edb03c8e4576d"
server_key_version: "encryptionkey@0"
I20260806 01:12:40.509104 22575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:40.527109 22575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:40.530985 22575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:40.531275 22575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:40.549703 22575 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45357
I20260806 01:12:40.549747 22671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45357 every 8 connection(s)
I20260806 01:12:40.551043 22575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:40.554641 22672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:40.556413 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22575
I20260806 01:12:40.556600 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:40.557199 22781 external_mini_cluster.cc:1428] Setting key 4fa8afcb16c381a21d0be7c48aaaab06
I20260806 01:12:40.569399 22672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a: Bootstrap starting.
I20260806 01:12:40.570988 22676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:40.559118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:42837 (local address 127.0.0.1:45357)
0806 01:12:40.559960 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:40.559971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:40.560011 (+ 40us) server_negotiation.cc:407] Connection header received
0806 01:12:40.561610 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:40.561636 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:40.562050 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:40.562383 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:40.563057 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:40.564434 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:40.566412 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:40.567181 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:40.567340 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:40.568676 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:40.568684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:40.568698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:40.569239 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:40.569316 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:40.569340 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:40.569478 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:40.569824 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 01:12:40.570039 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":386,"threads_started":1}
I20260806 01:12:40.572849 22672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:40.574195 22672 log.cc:824] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:40.574905 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:40.577004 22672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a: No bootstrap required, opened a new log
I20260806 01:12:40.586521 22676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:40.579398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:47961 (local address 127.0.0.1:45357)
0806 01:12:40.579584 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:40.579590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:40.580161 (+ 571us) server_negotiation.cc:407] Connection header received
0806 01:12:40.580315 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:40.580322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:40.580411 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:40.580529 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:40.581119 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:40.582345 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:40.583707 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:40.584474 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:40.584619 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:40.585669 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:40.585675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:40.585679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:40.585702 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:40.585752 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:40.585772 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:40.585902 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:40.586075 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 01:12:40.586128 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 01:12:40.586164 22672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } }
I20260806 01:12:40.586714 22672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:40.586882 22672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88eb47fb11ad4f5d8aafc4d884a2f44a, State: Initialized, Role: FOLLOWER
I20260806 01:12:40.587527 22672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } }
I20260806 01:12:40.587845 22672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:40.588009 22672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:40.588195 22672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:40.590358 22672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } }
I20260806 01:12:40.590857 22672 leader_election.cc:302] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88eb47fb11ad4f5d8aafc4d884a2f44a; no voters:
I20260806 01:12:40.591439 22672 leader_election.cc:288] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:40.591569 22684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:40.592855 22684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [term 1 LEADER]: Becoming Leader. State: Replica: 88eb47fb11ad4f5d8aafc4d884a2f44a, State: Running, Role: LEADER
I20260806 01:12:40.593535 22684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } }
I20260806 01:12:40.593781 22672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:40.596370 22685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } } }
I20260806 01:12:40.596807 22685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:40.596717 22687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88eb47fb11ad4f5d8aafc4d884a2f44a. Latest consensus state: current_term: 1 leader_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88eb47fb11ad4f5d8aafc4d884a2f44a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45357 } } }
I20260806 01:12:40.597007 22687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:40.599190 22691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:40.604828 22691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:40.613791 22691 catalog_manager.cc:1347] Generated new cluster ID: 54ec6064b0144398871502d7893dc7db
I20260806 01:12:40.613936 22691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:40.629612 22691 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:40.630988 22691 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:40.639605 22691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88eb47fb11ad4f5d8aafc4d884a2f44a: Generated new TSK 0
I20260806 01:12:40.640561 22691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:40.657282 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22575
W20260806 01:12:40.658910 22667 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.49:42837
W20260806 01:12:40.659830 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22575: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:40.659991 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22575
W20260806 01:12:40.686919 22556 connection.cc:535] client connection to 127.0.0.1:45357 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45357): Connection reset by peer (error 104)
2026-08-06T01:12:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2237 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:40 dist-test-slave-7f01 krb5kdc[22713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:40 dist-test-slave-7f01 krb5kdc[22713](info): set up 2 sockets
Aug 06 01:12:40 dist-test-slave-7f01 krb5kdc[22713](info): commencing operation
krb5kdc: starting...
W20260806 01:12:41.995074 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.006s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:42 dist-test-slave-7f01 krb5kdc[22713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:42.327016 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36679
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:42.680598 22734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:42.680959 22734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:42.681121 22734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:42.688402 22734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:42.688555 22734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:42.688652 22734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:42.688776 22734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:42.688854 22734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:42.698693 22734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:42.700114 22734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:42.701428 22734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:42.713483 22741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:42.713456 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:42.713613 22742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:42.714176 22734 server_base.cc:1034] running on GCE node
I20260806 01:12:42.715011 22734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:42.716405 22734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:42.717636 22734 hybrid_clock.cc:648] HybridClock initialized: now 1785978762717564 us; error 50 us; skew 500 ppm
I20260806 01:12:42.718031 22734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:42 dist-test-slave-7f01 krb5kdc[22713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978762, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:42.721217 22734 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:42.722599 22734 webserver.cc:460] Webserver started at http://127.0.0.1:34511/ using document root <none> and password file <none>
I20260806 01:12:42.723232 22734 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:42.723335 22734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:42.723578 22734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:42.726554 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "5d26d6b2540546a3af651cf1e1db8607"
format_stamp: "Formatted at 2026-08-06 01:12:42 on dist-test-slave-7f01"
server_key: "6007231ada1918998dac3e9640e9f757"
server_key_iv: "b8a7fb6b36f15e7bf12bfc89c96d9747"
server_key_version: "encryptionkey@0"
I20260806 01:12:42.727298 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "5d26d6b2540546a3af651cf1e1db8607"
format_stamp: "Formatted at 2026-08-06 01:12:42 on dist-test-slave-7f01"
server_key: "6007231ada1918998dac3e9640e9f757"
server_key_iv: "b8a7fb6b36f15e7bf12bfc89c96d9747"
server_key_version: "encryptionkey@0"
I20260806 01:12:42.732350 22734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260806 01:12:42.735623 22755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:42.736968 22734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:42.737162 22734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "5d26d6b2540546a3af651cf1e1db8607"
format_stamp: "Formatted at 2026-08-06 01:12:42 on dist-test-slave-7f01"
server_key: "6007231ada1918998dac3e9640e9f757"
server_key_iv: "b8a7fb6b36f15e7bf12bfc89c96d9747"
server_key_version: "encryptionkey@0"
I20260806 01:12:42.737355 22734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:42.743094 22734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:42.747228 22734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:42.747530 22734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:42.766443 22734 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36679
I20260806 01:12:42.766492 22844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36679 every 8 connection(s)
I20260806 01:12:42.767930 22734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:42.770047 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22734
I20260806 01:12:42.770241 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:42.770803 22781 external_mini_cluster.cc:1428] Setting key 4a2d0930f03332b3a78614bc6ac3dd7d
I20260806 01:12:42.772565 22845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:42.778669 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:42.772758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:34419 (local address 127.0.0.1:36679)
0806 01:12:42.773575 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:42.773587 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:42.773629 (+ 42us) server_negotiation.cc:407] Connection header received
0806 01:12:42.775198 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:42.775218 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:42.775565 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:42.775968 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:42.777106 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:42.777112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:42.777124 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:42.777138 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:42.777212 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:42.777232 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:42.777404 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:42.777513 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 01:12:42.777722 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":367,"threads_started":1}
I20260806 01:12:42.782354 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:42.785861 22845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607: Bootstrap starting.
I20260806 01:12:42.789878 22845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:42.790678 22850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:42.787930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:45901 (local address 127.0.0.1:36679)
0806 01:12:42.788145 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:42.788151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:42.788625 (+ 474us) server_negotiation.cc:407] Connection header received
0806 01:12:42.788765 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:42.788770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:42.788839 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:42.788952 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:42.789953 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:42.789958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:42.789964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:42.789983 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:42.790029 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:42.790056 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:42.790181 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:42.790327 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 01:12:42.790405 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:12:42.791275 22845 log.cc:824] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:42.794090 22845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607: No bootstrap required, opened a new log
I20260806 01:12:42.801357 22845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } }
I20260806 01:12:42.801733 22845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:42.801851 22845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26d6b2540546a3af651cf1e1db8607, State: Initialized, Role: FOLLOWER
I20260806 01:12:42.802518 22845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } }
I20260806 01:12:42.802749 22845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:42.802906 22845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:42.803081 22845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:42.805265 22845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } }
I20260806 01:12:42.805759 22845 leader_election.cc:302] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d26d6b2540546a3af651cf1e1db8607; no voters:
I20260806 01:12:42.806370 22845 leader_election.cc:288] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:42.806543 22861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:42.807305 22861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [term 1 LEADER]: Becoming Leader. State: Replica: 5d26d6b2540546a3af651cf1e1db8607, State: Running, Role: LEADER
I20260806 01:12:42.807997 22861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } }
I20260806 01:12:42.808866 22845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:42.810705 22865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d26d6b2540546a3af651cf1e1db8607. Latest consensus state: current_term: 1 leader_uuid: "5d26d6b2540546a3af651cf1e1db8607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } } }
I20260806 01:12:42.811059 22865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:42.811606 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d26d6b2540546a3af651cf1e1db8607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d26d6b2540546a3af651cf1e1db8607" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36679 } } }
I20260806 01:12:42.811982 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:42.812049 22871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:42.816974 22871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:42.826143 22871 catalog_manager.cc:1347] Generated new cluster ID: 6bd2e35890db43caa5a40875b24e9554
I20260806 01:12:42.826288 22871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:42.842322 22871 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:42.842769 22871 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:42.851936 22871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d26d6b2540546a3af651cf1e1db8607: Generated new TSK 0
I20260806 01:12:42.853005 22871 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:42.859444 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22734
W20260806 01:12:42.860605 22838 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.49:34419
W20260806 01:12:42.861380 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22734: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:42.861548 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22734
2026-08-06T01:12:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2203 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:42 dist-test-slave-7f01 krb5kdc[22887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:42 dist-test-slave-7f01 krb5kdc[22887](info): set up 2 sockets
Aug 06 01:12:42 dist-test-slave-7f01 krb5kdc[22887](info): commencing operation
krb5kdc: starting...
W20260806 01:12:44.195919 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:44 dist-test-slave-7f01 krb5kdc[22887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:44.542765 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:44.885535 22904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:44.885823 22904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:44.885915 22904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:44.893011 22904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:44.893118 22904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:44.893167 22904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:44.893227 22904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:44.893266 22904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:44.903249 22904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:44.904582 22904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:44.905827 22904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:44.917985 22913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:44.918103 22910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:44.918128 22911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:44.919073 22904 server_base.cc:1034] running on GCE node
I20260806 01:12:44.919931 22904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:44.921367 22904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:44.922575 22904 hybrid_clock.cc:648] HybridClock initialized: now 1785978764922538 us; error 30 us; skew 500 ppm
I20260806 01:12:44.922942 22904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:44 dist-test-slave-7f01 krb5kdc[22887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:44.930135 22904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:44.931730 22904 webserver.cc:460] Webserver started at http://127.0.0.1:35719/ using document root <none> and password file <none>
I20260806 01:12:44.932502 22904 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:44.932657 22904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:44.932976 22904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:44.936183 22904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "ba90d95b4d7343ac90cdac3388738ccc"
format_stamp: "Formatted at 2026-08-06 01:12:44 on dist-test-slave-7f01"
server_key: "610b03cbe559419b9ac9972eebc80565"
server_key_iv: "4e1301306b5d8ad52624cd38fcf9f62d"
server_key_version: "encryptionkey@0"
I20260806 01:12:44.936987 22904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "ba90d95b4d7343ac90cdac3388738ccc"
format_stamp: "Formatted at 2026-08-06 01:12:44 on dist-test-slave-7f01"
server_key: "610b03cbe559419b9ac9972eebc80565"
server_key_iv: "4e1301306b5d8ad52624cd38fcf9f62d"
server_key_version: "encryptionkey@0"
I20260806 01:12:44.942065 22904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:12:44.945583 22921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:44.946864 22904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 01:12:44.947062 22904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "ba90d95b4d7343ac90cdac3388738ccc"
format_stamp: "Formatted at 2026-08-06 01:12:44 on dist-test-slave-7f01"
server_key: "610b03cbe559419b9ac9972eebc80565"
server_key_iv: "4e1301306b5d8ad52624cd38fcf9f62d"
server_key_version: "encryptionkey@0"
I20260806 01:12:44.947263 22904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:44.958762 22904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:44.962728 22904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:44.963034 22904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:44.987526 22904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38341
I20260806 01:12:44.987571 22999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38341 every 8 connection(s)
I20260806 01:12:44.989771 22904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:44.995625 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 22904
I20260806 01:12:44.995862 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:44.996335 22781 external_mini_cluster.cc:1428] Setting key 4b2129e1cf736bb1b0e3bd04c1e22f4f
I20260806 01:12:44.996742 23000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 01:12:45 dist-test-slave-7f01 krb5kdc[22887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 01:12:45.015136 23000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc: Bootstrap starting.
I20260806 01:12:45.019634 23000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:45.021041 23000 log.cc:824] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:45.023073 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:44.998312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:47361 (local address 127.0.0.1:38341)
0806 01:12:44.999095 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:44.999111 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:44.999160 (+ 49us) server_negotiation.cc:407] Connection header received
0806 01:12:45.000899 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:45.000925 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:45.001383 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:45.003274 (+ 1891us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:45.003717 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:45.005182 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:45.006507 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:45.007238 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:45.007457 (+ 219us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:45.013162 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:45.013174 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:45.013195 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:45.013245 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:45.017722 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:45.018550 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:45.018571 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:45.018580 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:45.018801 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:45.019374 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:45.019382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:45.019388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:45.020235 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:45.020548 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:45.021200 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 01:12:45.021845 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":417,"threads_started":1}
I20260806 01:12:45.024605 23000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc: No bootstrap required, opened a new log
I20260806 01:12:45.027609 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:45.035440 23000 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } }
I20260806 01:12:45.035933 23000 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:45.036042 23000 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba90d95b4d7343ac90cdac3388738ccc, State: Initialized, Role: FOLLOWER
I20260806 01:12:45.036859 23000 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } }
I20260806 01:12:45.037135 23000 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:45.037282 23000 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:45.037449 23000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:45.040249 23000 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } }
I20260806 01:12:45.040841 23000 leader_election.cc:302] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba90d95b4d7343ac90cdac3388738ccc; no voters:
I20260806 01:12:45.041553 23000 leader_election.cc:288] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:45.041682 23010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:45.043273 23010 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [term 1 LEADER]: Becoming Leader. State: Replica: ba90d95b4d7343ac90cdac3388738ccc, State: Running, Role: LEADER
I20260806 01:12:45.043911 23010 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } }
I20260806 01:12:45.044171 23000 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:45.046639 23012 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba90d95b4d7343ac90cdac3388738ccc. Latest consensus state: current_term: 1 leader_uuid: "ba90d95b4d7343ac90cdac3388738ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } } }
I20260806 01:12:45.047912 23012 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:45.047402 23011 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba90d95b4d7343ac90cdac3388738ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba90d95b4d7343ac90cdac3388738ccc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38341 } } }
I20260806 01:12:45.048578 23011 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:45.052188 23018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:45.054508 23003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:45.033084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:51217 (local address 127.0.0.1:38341)
0806 01:12:45.033266 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:45.033271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:45.033698 (+ 427us) server_negotiation.cc:407] Connection header received
0806 01:12:45.033833 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:45.033841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:45.033926 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:45.034039 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:45.036346 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:45.037556 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:45.038785 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:45.039463 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:45.039591 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:45.042944 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:45.042950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:45.042955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:45.042995 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:45.049877 (+ 6882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:45.051363 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:45.051372 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:45.051378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:45.051598 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:45.052370 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:45.052378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:45.052384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:45.052919 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:45.053257 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:45.053676 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 01:12:45.053935 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 01:12:45.058725 23018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:45.068428 23018 catalog_manager.cc:1347] Generated new cluster ID: 6083f2d829b54560b376b60be4deb889
I20260806 01:12:45.068557 23018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:45.085898 23018 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:45.087567 23018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:45.098594 23018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba90d95b4d7343ac90cdac3388738ccc: Generated new TSK 0
I20260806 01:12:45.099419 23018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:45.109735 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 22904
I20260806 01:12:45.111709 22993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:45.203792 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 22904
W20260806 01:12:45.229575 22885 connection.cc:535] client connection to 127.0.0.1:38341 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38341): Connection reset by peer (error 104)
2026-08-06T01:12:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:45 dist-test-slave-7f01 krb5kdc[23041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:45 dist-test-slave-7f01 krb5kdc[23041](info): set up 2 sockets
Aug 06 01:12:45 dist-test-slave-7f01 krb5kdc[23041](info): commencing operation
krb5kdc: starting...
W20260806 01:12:46.543681 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:46 dist-test-slave-7f01 krb5kdc[23041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:46.895687 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33973
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:47.253062 23064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:47.253376 23064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:47.253470 23064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:47.260785 23064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:47.260910 23064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:47.260957 23064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:47.261018 23064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:47.261065 23064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:47.271108 23064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:47.272476 23064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:47.273784 23064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:47.286024 23072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:47.286002 23073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:47.285945 23075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:47.287122 23064 server_base.cc:1034] running on GCE node
I20260806 01:12:47.288079 23064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:47.289553 23064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:47.290838 23064 hybrid_clock.cc:648] HybridClock initialized: now 1785978767290785 us; error 68 us; skew 500 ppm
I20260806 01:12:47.291249 23064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:47 dist-test-slave-7f01 krb5kdc[23041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:47.294747 23064 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:47.296466 23064 webserver.cc:460] Webserver started at http://127.0.0.1:44265/ using document root <none> and password file <none>
I20260806 01:12:47.297201 23064 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:47.297363 23064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:47.297680 23064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:47.301084 23064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "c32c34320ac34aef9f65304d182d9d00"
format_stamp: "Formatted at 2026-08-06 01:12:47 on dist-test-slave-7f01"
server_key: "98f07fa1e78e970b2b703aa0c2a22163"
server_key_iv: "b8ab0168a8e8b2e094dc25dae94b561a"
server_key_version: "encryptionkey@0"
I20260806 01:12:47.301903 23064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "c32c34320ac34aef9f65304d182d9d00"
format_stamp: "Formatted at 2026-08-06 01:12:47 on dist-test-slave-7f01"
server_key: "98f07fa1e78e970b2b703aa0c2a22163"
server_key_iv: "b8ab0168a8e8b2e094dc25dae94b561a"
server_key_version: "encryptionkey@0"
I20260806 01:12:47.307263 23064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:12:47.310768 23087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:47.312282 23064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:47.312491 23064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "c32c34320ac34aef9f65304d182d9d00"
format_stamp: "Formatted at 2026-08-06 01:12:47 on dist-test-slave-7f01"
server_key: "98f07fa1e78e970b2b703aa0c2a22163"
server_key_iv: "b8ab0168a8e8b2e094dc25dae94b561a"
server_key_version: "encryptionkey@0"
I20260806 01:12:47.312708 23064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:47.342554 23064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:47.347054 23064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:47.347381 23064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:47.367102 23064 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33973
I20260806 01:12:47.367132 23170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33973 every 8 connection(s)
I20260806 01:12:47.368620 23064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:47.372004 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23064
I20260806 01:12:47.372220 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:47.372605 23172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:47.377892 22781 external_mini_cluster.cc:1428] Setting key b2da558bcda4bd21015a108ae8880b49
I20260806 01:12:47.383296 23172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00: Bootstrap starting.
I20260806 01:12:47.386889 23172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:47.388108 23172 log.cc:824] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:47.391690 23172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00: No bootstrap required, opened a new log
I20260806 01:12:47.392000 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:47.379817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:42579 (local address 127.0.0.1:33973)
0806 01:12:47.380676 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:47.380688 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:47.380733 (+ 45us) server_negotiation.cc:407] Connection header received
0806 01:12:47.382435 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:47.382458 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:47.382843 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:47.383164 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:47.383856 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:47.385194 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:47.387120 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:47.387860 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:47.388000 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:47.389391 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:47.389404 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:47.389424 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:47.390133 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:47.390206 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:47.390236 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:47.390349 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:47.390711 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 01:12:47.390918 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":395,"threads_started":1}
I20260806 01:12:47.395992 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:47.401710 23172 raft_consensus.cc:348] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } }
I20260806 01:12:47.402251 23172 raft_consensus.cc:374] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:47.402392 23172 raft_consensus.cc:729] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32c34320ac34aef9f65304d182d9d00, State: Initialized, Role: FOLLOWER
I20260806 01:12:47.403110 23172 consensus_queue.cc:260] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } }
I20260806 01:12:47.403453 23172 raft_consensus.cc:388] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:47.403626 23172 raft_consensus.cc:482] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:47.403903 23172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:47.407151 23172 raft_consensus.cc:504] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } }
I20260806 01:12:47.407868 23172 leader_election.cc:302] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c32c34320ac34aef9f65304d182d9d00; no voters:
I20260806 01:12:47.408660 23172 leader_election.cc:288] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:47.408767 23185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:47.409302 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:47.401761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:41815 (local address 127.0.0.1:33973)
0806 01:12:47.401986 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:47.401992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:47.402413 (+ 421us) server_negotiation.cc:407] Connection header received
0806 01:12:47.402631 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:47.402636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:47.402754 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:47.402879 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:47.403590 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:47.404589 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:47.406497 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:47.406956 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:47.407063 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:47.408342 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:47.408347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:47.408350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:47.408368 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:47.408431 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:47.408446 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:47.408603 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:47.408810 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 01:12:47.408867 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:12:47.410717 23185 raft_consensus.cc:686] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [term 1 LEADER]: Becoming Leader. State: Replica: c32c34320ac34aef9f65304d182d9d00, State: Running, Role: LEADER
I20260806 01:12:47.411310 23185 consensus_queue.cc:237] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } }
I20260806 01:12:47.412106 23172 sys_catalog.cc:564] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:47.414417 23187 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c32c34320ac34aef9f65304d182d9d00. Latest consensus state: current_term: 1 leader_uuid: "c32c34320ac34aef9f65304d182d9d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } } }
I20260806 01:12:47.414865 23187 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:47.419823 23186 sys_catalog.cc:455] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c32c34320ac34aef9f65304d182d9d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c32c34320ac34aef9f65304d182d9d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33973 } } }
I20260806 01:12:47.420109 23186 sys_catalog.cc:458] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:47.420542 23201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:47.426141 23201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:47.435990 23201 catalog_manager.cc:1347] Generated new cluster ID: f9b945e41ccf479e8f3a69446e5208fc
I20260806 01:12:47.436144 23201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:47.450044 23201 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:47.451876 23201 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:47.464148 23201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c32c34320ac34aef9f65304d182d9d00: Generated new TSK 0
I20260806 01:12:47.465041 23201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:47.476227 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23064
W20260806 01:12:47.477636 23163 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.49:42579
W20260806 01:12:47.478497 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23064: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:47.478669 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23064
W20260806 01:12:47.503268 23037 connection.cc:535] client connection to 127.0.0.1:33973 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33973): Connection reset by peer (error 104)
2026-08-06T01:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2273 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:47 dist-test-slave-7f01 krb5kdc[23216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:47 dist-test-slave-7f01 krb5kdc[23216](info): set up 2 sockets
Aug 06 01:12:47 dist-test-slave-7f01 krb5kdc[23216](info): commencing operation
krb5kdc: starting...
W20260806 01:12:48.804741 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.001s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:49 dist-test-slave-7f01 krb5kdc[23216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 01:12:49.156100 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:49.508010 23231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:49.508374 23231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:49.508570 23231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:49.515793 23231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:49.515918 23231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:49.515966 23231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:49.516021 23231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:49.516067 23231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:49.525862 23231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:49.527134 23231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:49.528487 23231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:49.540567 23238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:49.540664 23237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:49.540693 23240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:49.541339 23231 server_base.cc:1034] running on GCE node
I20260806 01:12:49.542043 23231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:49.543346 23231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:49.544565 23231 hybrid_clock.cc:648] HybridClock initialized: now 1785978769544504 us; error 53 us; skew 500 ppm
I20260806 01:12:49.544898 23231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:49 dist-test-slave-7f01 krb5kdc[23216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:49.548157 23231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 01:12:49.549563 23231 webserver.cc:460] Webserver started at http://127.0.0.1:46025/ using document root <none> and password file <none>
I20260806 01:12:49.550201 23231 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:49.550302 23231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:49.550554 23231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:49.553541 23231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "0571768216244eeabeca8b6f5bc71c6b"
format_stamp: "Formatted at 2026-08-06 01:12:49 on dist-test-slave-7f01"
server_key: "c335b57711d0c50f14c12f170d3ddef6"
server_key_iv: "dba05e64609966cded6a157348263633"
server_key_version: "encryptionkey@0"
I20260806 01:12:49.554262 23231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "0571768216244eeabeca8b6f5bc71c6b"
format_stamp: "Formatted at 2026-08-06 01:12:49 on dist-test-slave-7f01"
server_key: "c335b57711d0c50f14c12f170d3ddef6"
server_key_iv: "dba05e64609966cded6a157348263633"
server_key_version: "encryptionkey@0"
I20260806 01:12:49.559320 23231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:12:49.562934 23252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:49.564395 23231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:49.564589 23231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "0571768216244eeabeca8b6f5bc71c6b"
format_stamp: "Formatted at 2026-08-06 01:12:49 on dist-test-slave-7f01"
server_key: "c335b57711d0c50f14c12f170d3ddef6"
server_key_iv: "dba05e64609966cded6a157348263633"
server_key_version: "encryptionkey@0"
I20260806 01:12:49.564764 23231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:49.570540 23231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:49.574954 23231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:49.575289 23231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:49.594208 23231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32963
I20260806 01:12:49.594249 23333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32963 every 8 connection(s)
I20260806 01:12:49.595805 23231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:49.599548 23334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:49.600472 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23231
I20260806 01:12:49.600670 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:49.601120 22781 external_mini_cluster.cc:1428] Setting key e91f9f5d3bfaef253eeb053d2717f4dc
I20260806 01:12:49.609145 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:49.602921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:41597 (local address 127.0.0.1:32963)
0806 01:12:49.603715 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:49.603775 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:49.603830 (+ 55us) server_negotiation.cc:407] Connection header received
0806 01:12:49.605588 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:49.605621 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:49.606001 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:49.606334 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:49.607487 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:49.607493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:49.607505 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:49.607519 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:49.607571 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:49.607594 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:49.607820 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:49.607921 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 01:12:49.608123 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":361,"threads_started":1}
I20260806 01:12:49.613049 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:49.613507 23334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b: Bootstrap starting.
I20260806 01:12:49.616603 23334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:49.617811 23334 log.cc:824] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:49.620430 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:49.617676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:34185 (local address 127.0.0.1:32963)
0806 01:12:49.617901 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:49.617906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:49.618357 (+ 451us) server_negotiation.cc:407] Connection header received
0806 01:12:49.618578 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:49.618586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:49.618653 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:49.618776 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:49.619701 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:49.619705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:49.619708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:49.619722 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:49.619814 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:49.619832 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:49.619919 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:49.620077 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 01:12:49.620125 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 01:12:49.621295 23334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b: No bootstrap required, opened a new log
I20260806 01:12:49.628700 23334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } }
I20260806 01:12:49.629093 23334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:49.629211 23334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0571768216244eeabeca8b6f5bc71c6b, State: Initialized, Role: FOLLOWER
I20260806 01:12:49.629827 23334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } }
I20260806 01:12:49.630084 23334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:49.630224 23334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:49.630402 23334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:49.632603 23334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } }
I20260806 01:12:49.633090 23334 leader_election.cc:302] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0571768216244eeabeca8b6f5bc71c6b; no voters:
I20260806 01:12:49.633764 23334 leader_election.cc:288] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:49.633886 23344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:49.634552 23344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [term 1 LEADER]: Becoming Leader. State: Replica: 0571768216244eeabeca8b6f5bc71c6b, State: Running, Role: LEADER
I20260806 01:12:49.635109 23344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } }
I20260806 01:12:49.636183 23334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:49.638422 23347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0571768216244eeabeca8b6f5bc71c6b. Latest consensus state: current_term: 1 leader_uuid: "0571768216244eeabeca8b6f5bc71c6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } } }
I20260806 01:12:49.638938 23347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:49.638572 23346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0571768216244eeabeca8b6f5bc71c6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0571768216244eeabeca8b6f5bc71c6b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32963 } } }
I20260806 01:12:49.639117 23346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:49.640952 23354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:49.646207 23354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:49.655347 23354 catalog_manager.cc:1347] Generated new cluster ID: e412b6895fc9402bb76708069cf2b36d
I20260806 01:12:49.655475 23354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:49.678956 23354 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:49.679493 23354 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:49.696403 23354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0571768216244eeabeca8b6f5bc71c6b: Generated new TSK 0
I20260806 01:12:49.697454 23354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:49.712203 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23231
W20260806 01:12:49.713326 23320 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.49:41597
W20260806 01:12:49.714030 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23231: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:49.714184 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23231
2026-08-06T01:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2236 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:49 dist-test-slave-7f01 krb5kdc[23374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:49 dist-test-slave-7f01 krb5kdc[23374](info): set up 2 sockets
Aug 06 01:12:49 dist-test-slave-7f01 krb5kdc[23374](info): commencing operation
krb5kdc: starting...
W20260806 01:12:51.064451 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:51 dist-test-slave-7f01 krb5kdc[23374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:51.415706 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42509
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:42509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:51.762301 23395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:51.762617 23395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:51.762786 23395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:51.770051 23395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:51.770167 23395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:51.770309 23395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:51.770423 23395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:51.770484 23395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:51.780330 23395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:42509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:51.781690 23395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:51.782997 23395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:51.795070 23405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:51.795213 23402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:51.795189 23403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:51.796047 23395 server_base.cc:1034] running on GCE node
I20260806 01:12:51.796926 23395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:51.798358 23395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:51.799598 23395 hybrid_clock.cc:648] HybridClock initialized: now 1785978771799565 us; error 59 us; skew 500 ppm
I20260806 01:12:51.800011 23395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:51 dist-test-slave-7f01 krb5kdc[23374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978771, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:51.807157 23395 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:51.808811 23395 webserver.cc:460] Webserver started at http://127.22.63.126:35207/ using document root <none> and password file <none>
I20260806 01:12:51.809525 23395 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:51.809675 23395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:51.809995 23395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:51.813189 23395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "da2571c73ec040da8803243b16c6f9fd"
format_stamp: "Formatted at 2026-08-06 01:12:51 on dist-test-slave-7f01"
server_key: "69b8e41fea6195df75349cb624ae08b9"
server_key_iv: "910da5c065cf33436c7f04eae4b37641"
server_key_version: "encryptionkey@0"
I20260806 01:12:51.814059 23395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "da2571c73ec040da8803243b16c6f9fd"
format_stamp: "Formatted at 2026-08-06 01:12:51 on dist-test-slave-7f01"
server_key: "69b8e41fea6195df75349cb624ae08b9"
server_key_iv: "910da5c065cf33436c7f04eae4b37641"
server_key_version: "encryptionkey@0"
I20260806 01:12:51.819401 23395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 01:12:51.823052 23414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:51.824689 23395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 01:12:51.824898 23395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "da2571c73ec040da8803243b16c6f9fd"
format_stamp: "Formatted at 2026-08-06 01:12:51 on dist-test-slave-7f01"
server_key: "69b8e41fea6195df75349cb624ae08b9"
server_key_iv: "910da5c065cf33436c7f04eae4b37641"
server_key_version: "encryptionkey@0"
I20260806 01:12:51.825131 23395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:51.843789 23395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:51.847994 23395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:51.848300 23395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:51.867651 23395 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:42509
I20260806 01:12:51.867681 23493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:42509 every 8 connection(s)
I20260806 01:12:51.869287 23395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:51.873082 23494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:51.879209 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23395
I20260806 01:12:51.879426 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:51.880086 22781 external_mini_cluster.cc:1428] Setting key 4392ce35c04bbff55f1eb69c0e842293
I20260806 01:12:51.888500 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd: Bootstrap starting.
I20260806 01:12:51.891639 23494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:51.892726 23494 log.cc:824] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:51.896363 23494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd: No bootstrap required, opened a new log
Aug 06 01:12:51 dist-test-slave-7f01 krb5kdc[23374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:51.905077 23494 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } }
I20260806 01:12:51.905541 23494 raft_consensus.cc:374] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:51.905668 23494 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2571c73ec040da8803243b16c6f9fd, State: Initialized, Role: FOLLOWER
I20260806 01:12:51.906376 23494 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } }
I20260806 01:12:51.906690 23494 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:51.906867 23494 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:51.907064 23494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:51.908458 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:51.882119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34613 (local address 127.22.63.126:42509)
0806 01:12:51.886463 (+ 4344us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:51.886482 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:51.886536 (+ 54us) server_negotiation.cc:407] Connection header received
0806 01:12:51.888507 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:51.888531 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:51.889023 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:51.889410 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:51.891028 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:51.892409 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:51.894149 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:51.895250 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:51.895529 (+ 279us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:51.900284 (+ 4755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:51.900295 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:51.900311 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:51.900360 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:51.903571 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:51.904553 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:51.904559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:51.904566 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:51.904726 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:51.905226 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:51.905229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:51.905231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:51.905912 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:51.906254 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:51.906893 (+ 639us) server_negotiation.cc:299] Negotiation successful
0806 01:12:51.907297 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4082,"thread_start_us":3614,"threads_started":1}
I20260806 01:12:51.909585 23494 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } }
I20260806 01:12:51.910081 23494 leader_election.cc:302] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2571c73ec040da8803243b16c6f9fd; no voters:
I20260806 01:12:51.910797 23494 leader_election.cc:288] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:51.910918 23500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:51.912528 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:51.913463 23494 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:51.917852 23500 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [term 1 LEADER]: Becoming Leader. State: Replica: da2571c73ec040da8803243b16c6f9fd, State: Running, Role: LEADER
I20260806 01:12:51.918586 23500 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } }
I20260806 01:12:51.923039 23501 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2571c73ec040da8803243b16c6f9fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } } }
I20260806 01:12:51.923067 23502 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2571c73ec040da8803243b16c6f9fd. Latest consensus state: current_term: 1 leader_uuid: "da2571c73ec040da8803243b16c6f9fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2571c73ec040da8803243b16c6f9fd" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 42509 } } }
I20260806 01:12:51.923559 23501 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:51.923607 23502 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:51.939426 23524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:51.940196 23498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:51.917755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41931 (local address 127.22.63.126:42509)
0806 01:12:51.918004 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:51.918011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:51.918546 (+ 535us) server_negotiation.cc:407] Connection header received
0806 01:12:51.918761 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:51.918766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:51.918847 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:51.918962 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:51.921402 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:51.922712 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:51.925172 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:51.926156 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:51.926316 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:51.931081 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:51.931091 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:51.931099 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:51.931151 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:51.935978 (+ 4827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:51.937221 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:51.937226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:51.937229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:51.937360 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:51.937887 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:51.937892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:51.937895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:51.938278 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:51.938587 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:51.939482 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 01:12:51.939701 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:12:51.945441 23524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:51.954456 23524 catalog_manager.cc:1347] Generated new cluster ID: 5d5894d5081c4927bf631ee20fea2210
I20260806 01:12:51.954592 23524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:51.962755 23524 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:51.964177 23524 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:51.973781 23524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da2571c73ec040da8803243b16c6f9fd: Generated new TSK 0
I20260806 01:12:51.974833 23524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:51.984498 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23395
I20260806 01:12:51.985670 23483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:52.075639 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23395
W20260806 01:12:52.099501 23371 connection.cc:535] client connection to 127.22.63.126:42509 recv error: Network error: failed to read from TLS socket (remote: 127.22.63.126:42509): Connection reset by peer (error 104)
2026-08-06T01:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:52 dist-test-slave-7f01 krb5kdc[23540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:52 dist-test-slave-7f01 krb5kdc[23540](info): set up 2 sockets
Aug 06 01:12:52 dist-test-slave-7f01 krb5kdc[23540](info): commencing operation
krb5kdc: starting...
W20260806 01:12:53.408614 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.002s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:53 dist-test-slave-7f01 krb5kdc[23540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:53.750943 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33723
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:38125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:33723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:54.107545 23558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:54.107870 23558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:54.108029 23558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:54.115330 23558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:54.115437 23558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:54.115545 23558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:54.115626 23558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:54.115710 23558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:54.125890 23558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:38125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:33723
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:33723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:54.127249 23558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:54.128607 23558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:54.140477 23564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:54.140547 23566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:54.140550 23563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:54.141775 23558 server_base.cc:1034] running on GCE node
I20260806 01:12:54.142592 23558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:54.144104 23558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:54.145316 23558 hybrid_clock.cc:648] HybridClock initialized: now 1785978774145280 us; error 35 us; skew 500 ppm
I20260806 01:12:54.145696 23558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:54 dist-test-slave-7f01 krb5kdc[23540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978774, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:54.149034 23558 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:54.150630 23558 webserver.cc:460] Webserver started at http://127.22.63.126:35489/ using document root <none> and password file <none>
I20260806 01:12:54.151360 23558 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:54.151513 23558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:54.151909 23558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:54.154951 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "3f036c8063d44331b566f4c0463bf89f"
format_stamp: "Formatted at 2026-08-06 01:12:54 on dist-test-slave-7f01"
server_key: "f7a91b3b2d58df9b004ba7cdbd2e6dea"
server_key_iv: "65a8fa00ba714f5e4841a9e1a165671c"
server_key_version: "encryptionkey@0"
I20260806 01:12:54.155813 23558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "3f036c8063d44331b566f4c0463bf89f"
format_stamp: "Formatted at 2026-08-06 01:12:54 on dist-test-slave-7f01"
server_key: "f7a91b3b2d58df9b004ba7cdbd2e6dea"
server_key_iv: "65a8fa00ba714f5e4841a9e1a165671c"
server_key_version: "encryptionkey@0"
I20260806 01:12:54.160910 23558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:12:54.164381 23577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:54.165692 23558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:54.165906 23558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "3f036c8063d44331b566f4c0463bf89f"
format_stamp: "Formatted at 2026-08-06 01:12:54 on dist-test-slave-7f01"
server_key: "f7a91b3b2d58df9b004ba7cdbd2e6dea"
server_key_iv: "65a8fa00ba714f5e4841a9e1a165671c"
server_key_version: "encryptionkey@0"
I20260806 01:12:54.166106 23558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:54.187371 23558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:54.191684 23558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:54.192006 23558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:54.210599 23558 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:33723
I20260806 01:12:54.210631 23658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:33723 every 8 connection(s)
I20260806 01:12:54.211947 23558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:54.215523 23659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:54.216238 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23558
I20260806 01:12:54.216429 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:54.217048 22781 external_mini_cluster.cc:1428] Setting key dd8331110772f5b12a618de7970447c0
I20260806 01:12:54.229422 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f: Bootstrap starting.
I20260806 01:12:54.231492 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:54.218818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48705 (local address 127.22.63.126:33723)
0806 01:12:54.219714 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:54.219725 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:54.219833 (+ 108us) server_negotiation.cc:407] Connection header received
0806 01:12:54.221894 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:54.221917 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:54.222294 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:54.222657 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:54.223203 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:54.224526 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:54.226546 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:54.227308 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:54.227471 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:54.228855 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:54.228863 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:54.228875 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:54.229571 (+ 696us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:54.229675 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:54.229735 (+ 60us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:54.229919 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:54.230374 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 01:12:54.230532 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":434,"threads_started":1}
I20260806 01:12:54.233011 23659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:54.234157 23659 log.cc:824] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:54.235368 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:54.237471 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f: No bootstrap required, opened a new log
I20260806 01:12:54.247838 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:54.240862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34749 (local address 127.22.63.126:33723)
0806 01:12:54.241062 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:54.241069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:54.241515 (+ 446us) server_negotiation.cc:407] Connection header received
0806 01:12:54.241673 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:54.241681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:54.241763 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:54.241882 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:54.242449 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:54.243660 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:54.245077 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:54.245814 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:54.245968 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:54.247014 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:54.247021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:54.247025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:54.247047 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:54.247111 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:54.247140 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:54.247262 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:54.247439 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 01:12:54.247495 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:12:54.247401 23659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } }
I20260806 01:12:54.248026 23659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:54.248159 23659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f036c8063d44331b566f4c0463bf89f, State: Initialized, Role: FOLLOWER
I20260806 01:12:54.248898 23659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } }
I20260806 01:12:54.249194 23659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:54.249347 23659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:54.249547 23659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:54.251721 23659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } }
I20260806 01:12:54.252345 23659 leader_election.cc:302] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f036c8063d44331b566f4c0463bf89f; no voters:
I20260806 01:12:54.252947 23659 leader_election.cc:288] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:54.253115 23673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:54.253811 23673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [term 1 LEADER]: Becoming Leader. State: Replica: 3f036c8063d44331b566f4c0463bf89f, State: Running, Role: LEADER
I20260806 01:12:54.254379 23673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } }
I20260806 01:12:54.255128 23659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:54.257732 23675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f036c8063d44331b566f4c0463bf89f. Latest consensus state: current_term: 1 leader_uuid: "3f036c8063d44331b566f4c0463bf89f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } } }
I20260806 01:12:54.258519 23675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:54.257975 23674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f036c8063d44331b566f4c0463bf89f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f036c8063d44331b566f4c0463bf89f" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 33723 } } }
I20260806 01:12:54.259222 23674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:54.259663 23687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:54.265988 23687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:54.277104 23687 catalog_manager.cc:1347] Generated new cluster ID: 65a61b67d24142418672449c90b93ed8
I20260806 01:12:54.277252 23687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:54.317090 23687 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:54.318418 23687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:54.328775 23687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f036c8063d44331b566f4c0463bf89f: Generated new TSK 0
I20260806 01:12:54.329602 23687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:54.342682 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23558
W20260806 01:12:54.344131 23650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48705
W20260806 01:12:54.344954 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23558: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:54.345130 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23558
W20260806 01:12:54.370601 23537 connection.cc:535] client connection to 127.22.63.126:33723 recv error: Network error: failed to read from TLS socket (remote: 127.22.63.126:33723): Connection reset by peer (error 104)
2026-08-06T01:12:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:54 dist-test-slave-7f01 krb5kdc[23701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:54 dist-test-slave-7f01 krb5kdc[23701](info): set up 2 sockets
Aug 06 01:12:54 dist-test-slave-7f01 krb5kdc[23701](info): commencing operation
krb5kdc: starting...
W20260806 01:12:55.708406 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:55 dist-test-slave-7f01 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:56.056061 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40119
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:40119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:56.400944 23721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:56.401222 23721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:56.401314 23721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:56.408334 23721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:56.408433 23721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:56.408499 23721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:56.408563 23721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:56.408608 23721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:56.418207 23721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:40119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:40119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:56.419397 23721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:56.420639 23721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:56.432528 23726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:56.432765 23730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:56.433573 23721 server_base.cc:1034] running on GCE node
W20260806 01:12:56.433842 23727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:56.434679 23721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:56.436048 23721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:56.437247 23721 hybrid_clock.cc:648] HybridClock initialized: now 1785978776437210 us; error 28 us; skew 500 ppm
I20260806 01:12:56.437583 23721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:56 dist-test-slave-7f01 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978776, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:56.440605 23721 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:56.442044 23721 webserver.cc:460] Webserver started at http://127.22.63.126:32779/ using document root <none> and password file <none>
I20260806 01:12:56.442737 23721 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:56.442890 23721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:56.443210 23721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:56.446221 23721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "430e7f2e94b04cdca07dcff36083e578"
format_stamp: "Formatted at 2026-08-06 01:12:56 on dist-test-slave-7f01"
server_key: "354b6790883c37daf23cb7529287d767"
server_key_iv: "341c9cac9adc023611ee8a553a056bec"
server_key_version: "encryptionkey@0"
I20260806 01:12:56.447003 23721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "430e7f2e94b04cdca07dcff36083e578"
format_stamp: "Formatted at 2026-08-06 01:12:56 on dist-test-slave-7f01"
server_key: "354b6790883c37daf23cb7529287d767"
server_key_iv: "341c9cac9adc023611ee8a553a056bec"
server_key_version: "encryptionkey@0"
I20260806 01:12:56.451997 23721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 01:12:56.455227 23738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:56.456550 23721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:12:56.456734 23721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "430e7f2e94b04cdca07dcff36083e578"
format_stamp: "Formatted at 2026-08-06 01:12:56 on dist-test-slave-7f01"
server_key: "354b6790883c37daf23cb7529287d767"
server_key_iv: "341c9cac9adc023611ee8a553a056bec"
server_key_version: "encryptionkey@0"
I20260806 01:12:56.456936 23721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:56.462495 23721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:56.466348 23721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:56.466651 23721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:56.484907 23721 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:40119
I20260806 01:12:56.484942 23822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:40119 every 8 connection(s)
I20260806 01:12:56.486304 23721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:56.489337 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23721
I20260806 01:12:56.489521 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:56.490129 22781 external_mini_cluster.cc:1428] Setting key 1f614dbaa2161df0d8169d78b8adfd4d
I20260806 01:12:56.490334 23823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:56.498174 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:56.491906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44963 (local address 127.22.63.126:40119)
0806 01:12:56.492679 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:56.492690 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:56.492728 (+ 38us) server_negotiation.cc:407] Connection header received
0806 01:12:56.494887 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:56.494907 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:56.495212 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:56.495603 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:56.496674 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:56.496681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:56.496692 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:56.496711 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:56.496761 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:56.496778 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:56.497036 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:56.497184 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 01:12:56.497345 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":341,"threads_started":1}
I20260806 01:12:56.501793 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:56.503333 23823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578: Bootstrap starting.
I20260806 01:12:56.506559 23823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:56.507946 23823 log.cc:824] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:56.508777 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:56.506278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43517 (local address 127.22.63.126:40119)
0806 01:12:56.506464 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:56.506471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:56.506825 (+ 354us) server_negotiation.cc:407] Connection header received
0806 01:12:56.506969 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:56.506977 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:56.507064 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:56.507179 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:56.508106 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:56.508111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:56.508115 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:12:56.508134 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:56.508177 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:12:56.508197 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:56.508319 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:56.508398 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 01:12:56.508465 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:12:56.510954 23823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578: No bootstrap required, opened a new log
I20260806 01:12:56.518106 23823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } }
I20260806 01:12:56.518440 23823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:56.518524 23823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430e7f2e94b04cdca07dcff36083e578, State: Initialized, Role: FOLLOWER
I20260806 01:12:56.518996 23823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } }
I20260806 01:12:56.519189 23823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:56.519289 23823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:56.519446 23823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:56.521915 23823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } }
I20260806 01:12:56.522418 23823 leader_election.cc:302] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 430e7f2e94b04cdca07dcff36083e578; no voters:
I20260806 01:12:56.522934 23823 leader_election.cc:288] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:56.523083 23836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:56.523782 23836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [term 1 LEADER]: Becoming Leader. State: Replica: 430e7f2e94b04cdca07dcff36083e578, State: Running, Role: LEADER
I20260806 01:12:56.524340 23836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } }
I20260806 01:12:56.525202 23823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:56.527098 23839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 430e7f2e94b04cdca07dcff36083e578. Latest consensus state: current_term: 1 leader_uuid: "430e7f2e94b04cdca07dcff36083e578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } } }
I20260806 01:12:56.527494 23839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:56.527318 23838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "430e7f2e94b04cdca07dcff36083e578" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "430e7f2e94b04cdca07dcff36083e578" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 40119 } } }
I20260806 01:12:56.527714 23838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578 [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:56.529664 23844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:56.535194 23844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:56.543998 23844 catalog_manager.cc:1347] Generated new cluster ID: 5d40de7661f740098a440248e648786e
I20260806 01:12:56.544137 23844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:56.553508 23844 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:56.553969 23844 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:56.563062 23844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 430e7f2e94b04cdca07dcff36083e578: Generated new TSK 0
I20260806 01:12:56.564049 23844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:56.570016 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23721
W20260806 01:12:56.571100 23815 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44963
W20260806 01:12:56.571830 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:12:56.571995 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23721
2026-08-06T01:12:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2229 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:56 dist-test-slave-7f01 krb5kdc[23863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:56 dist-test-slave-7f01 krb5kdc[23863](info): set up 2 sockets
Aug 06 01:12:56 dist-test-slave-7f01 krb5kdc[23863](info): commencing operation
krb5kdc: starting...
W20260806 01:12:57.916096 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:12:58 dist-test-slave-7f01 krb5kdc[23863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:12:58.249480 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:35953
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:35953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:12:58.603121 23884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:12:58.603418 23884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:12:58.603578 23884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:12:58.610991 23884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:12:58.611105 23884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:12:58.611209 23884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:12:58.611315 23884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:12:58.611402 23884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:12:58.621735 23884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:35953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:35953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:12:58.623127 23884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:12:58.624550 23884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:12:58.636933 23891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:58.636852 23894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:12:58.637068 23892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:12:58.638660 23884 server_base.cc:1034] running on GCE node
I20260806 01:12:58.639653 23884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:12:58.641498 23884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:12:58.642740 23884 hybrid_clock.cc:648] HybridClock initialized: now 1785978778642677 us; error 53 us; skew 500 ppm
I20260806 01:12:58.643142 23884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:12:58 dist-test-slave-7f01 krb5kdc[23863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:12:58.650990 23884 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:12:58.654681 23884 webserver.cc:460] Webserver started at http://127.22.63.126:45453/ using document root <none> and password file <none>
I20260806 01:12:58.655463 23884 fs_manager.cc:359] Metadata directory not provided
I20260806 01:12:58.655598 23884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:12:58.655987 23884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:12:58.659281 23884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "710145905a744de4878b2bf059fc1f0e"
format_stamp: "Formatted at 2026-08-06 01:12:58 on dist-test-slave-7f01"
server_key: "fce157681b25d7e97027c303fa57016d"
server_key_iv: "3c2ce1391cc2646f25be4932e57d243a"
server_key_version: "encryptionkey@0"
I20260806 01:12:58.660219 23884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "710145905a744de4878b2bf059fc1f0e"
format_stamp: "Formatted at 2026-08-06 01:12:58 on dist-test-slave-7f01"
server_key: "fce157681b25d7e97027c303fa57016d"
server_key_iv: "3c2ce1391cc2646f25be4932e57d243a"
server_key_version: "encryptionkey@0"
I20260806 01:12:58.665503 23884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:12:58.669061 23904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:12:58.670353 23884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:12:58.670549 23884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "710145905a744de4878b2bf059fc1f0e"
format_stamp: "Formatted at 2026-08-06 01:12:58 on dist-test-slave-7f01"
server_key: "fce157681b25d7e97027c303fa57016d"
server_key_iv: "3c2ce1391cc2646f25be4932e57d243a"
server_key_version: "encryptionkey@0"
I20260806 01:12:58.670740 23884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:12:58.690330 23884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:12:58.694447 23884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:12:58.694736 23884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:12:58.713779 23884 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:35953
I20260806 01:12:58.713783 23981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:35953 every 8 connection(s)
I20260806 01:12:58.715225 23884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:12:58.718978 23982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:12:58.719058 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 23884
I20260806 01:12:58.719269 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:12:58.719705 22781 external_mini_cluster.cc:1428] Setting key d6cb7d42310ffdc35a0de929d07d2b47
I20260806 01:12:58.733652 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e: Bootstrap starting.
Aug 06 01:12:58 dist-test-slave-7f01 krb5kdc[23863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:12:58.738013 23982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e: Neither blocks nor log segments found. Creating new log.
I20260806 01:12:58.739126 23982 log.cc:824] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e: Log is configured to *not* fsync() on all Append() calls
I20260806 01:12:58.742450 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e: No bootstrap required, opened a new log
I20260806 01:12:58.743904 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:58.721433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58207 (local address 127.22.63.126:35953)
0806 01:12:58.722312 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:58.722323 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:58.722373 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:12:58.723957 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:58.723979 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:58.724373 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:58.724693 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:58.726515 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:58.727867 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:58.729701 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:58.730415 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:58.730556 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:58.736010 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:58.736020 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:58.736039 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:58.736095 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:58.739477 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:58.740366 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:58.740378 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:58.740389 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:58.740598 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:58.741070 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:58.741075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:58.741077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:58.741629 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:58.741857 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:58.742378 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 01:12:58.742835 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":447,"threads_started":1}
I20260806 01:12:58.747669 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:12:58.751926 23982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } }
I20260806 01:12:58.752411 23982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:12:58.752521 23982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710145905a744de4878b2bf059fc1f0e, State: Initialized, Role: FOLLOWER
I20260806 01:12:58.753151 23982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } }
I20260806 01:12:58.753432 23982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:12:58.753602 23982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:12:58.753813 23982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:12:58.756949 23982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } }
I20260806 01:12:58.757544 23982 leader_election.cc:302] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 710145905a744de4878b2bf059fc1f0e; no voters:
I20260806 01:12:58.758186 23982 leader_election.cc:288] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:12:58.758294 23995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:12:58.759608 23995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [term 1 LEADER]: Becoming Leader. State: Replica: 710145905a744de4878b2bf059fc1f0e, State: Running, Role: LEADER
I20260806 01:12:58.760295 23995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } }
I20260806 01:12:58.760680 23982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:12:58.764146 23998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 710145905a744de4878b2bf059fc1f0e. Latest consensus state: current_term: 1 leader_uuid: "710145905a744de4878b2bf059fc1f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } } }
I20260806 01:12:58.764582 23998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:58.764343 23996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "710145905a744de4878b2bf059fc1f0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710145905a744de4878b2bf059fc1f0e" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 35953 } } }
I20260806 01:12:58.765218 23996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e [sys.catalog]: This master's current role is: LEADER
I20260806 01:12:58.774253 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:12:58.753330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58681 (local address 127.22.63.126:35953)
0806 01:12:58.753539 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 01:12:58.753546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:12:58.754022 (+ 476us) server_negotiation.cc:407] Connection header received
0806 01:12:58.754211 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:12:58.754215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:12:58.754275 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:12:58.754359 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:12:58.756260 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:58.756977 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:58.758789 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:12:58.759776 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:12:58.759916 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:12:58.763399 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:12:58.763406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:12:58.763410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:12:58.763451 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:12:58.769093 (+ 5642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:58.770208 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:58.770216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:58.770223 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:58.770400 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:12:58.770996 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:12:58.771003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:12:58.771009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:12:58.771537 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:12:58.773219 (+ 1682us) server_negotiation.cc:1035] Waiting for connection context
0806 01:12:58.773410 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:12:58.773717 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:12:58.779207 24005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:12:58.785720 24005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:12:58.794528 24005 catalog_manager.cc:1347] Generated new cluster ID: 58f8b85c7a7f4eb9a2d288bbc03726e0
I20260806 01:12:58.794651 24005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:12:58.810355 24005 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:12:58.812347 24005 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:12:58.827072 24005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 710145905a744de4878b2bf059fc1f0e: Generated new TSK 0
I20260806 01:12:58.828049 24005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:12:58.839599 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 23884
I20260806 01:12:58.840682 23976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:12:58.932905 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 23884
2026-08-06T01:12:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:12:59 dist-test-slave-7f01 krb5kdc[24023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:12:59 dist-test-slave-7f01 krb5kdc[24023](info): set up 2 sockets
Aug 06 01:12:59 dist-test-slave-7f01 krb5kdc[24023](info): commencing operation
krb5kdc: starting...
W20260806 01:13:00.274688 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:13:00 dist-test-slave-7f01 krb5kdc[24023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:13:00.623499 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:39883
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:34141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:39883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:00.970432 24040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:00.970775 24040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:00.970959 24040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:00.978286 24040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:00.978446 24040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:00.978559 24040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:00.978664 24040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:00.978752 24040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:00.989555 24040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:34141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:39883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:39883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:00.990994 24040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:00.992350 24040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:01.004700 24050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:01.004825 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:01.004730 24057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:01.005729 24040 server_base.cc:1034] running on GCE node
I20260806 01:13:01.006551 24040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:01.008121 24040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:01.009357 24040 hybrid_clock.cc:648] HybridClock initialized: now 1785978781009321 us; error 49 us; skew 500 ppm
I20260806 01:13:01.009728 24040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:01 dist-test-slave-7f01 krb5kdc[24023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978781, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:01.013078 24040 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:13:01.014647 24040 webserver.cc:460] Webserver started at http://127.22.63.126:32873/ using document root <none> and password file <none>
I20260806 01:13:01.015374 24040 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:01.015528 24040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:01.015923 24040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:01.019032 24040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "ff8925586ae748d79356111924aeecd4"
format_stamp: "Formatted at 2026-08-06 01:13:01 on dist-test-slave-7f01"
server_key: "81587a6f894507c96c136ca761675e1e"
server_key_iv: "bc3db28e5a0e6e9bdb609c3d1367c3c2"
server_key_version: "encryptionkey@0"
I20260806 01:13:01.019886 24040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "ff8925586ae748d79356111924aeecd4"
format_stamp: "Formatted at 2026-08-06 01:13:01 on dist-test-slave-7f01"
server_key: "81587a6f894507c96c136ca761675e1e"
server_key_iv: "bc3db28e5a0e6e9bdb609c3d1367c3c2"
server_key_version: "encryptionkey@0"
I20260806 01:13:01.025031 24040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:13:01.028584 24067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:01.029937 24040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260806 01:13:01.030145 24040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "ff8925586ae748d79356111924aeecd4"
format_stamp: "Formatted at 2026-08-06 01:13:01 on dist-test-slave-7f01"
server_key: "81587a6f894507c96c136ca761675e1e"
server_key_iv: "bc3db28e5a0e6e9bdb609c3d1367c3c2"
server_key_version: "encryptionkey@0"
I20260806 01:13:01.030372 24040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:01.045264 24040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:01.049501 24040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:01.049817 24040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:01.069044 24040 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:39883
I20260806 01:13:01.069077 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:39883 every 8 connection(s)
I20260806 01:13:01.070534 24040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:01.074294 24130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:01.078617 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24040
I20260806 01:13:01.078809 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:13:01.079376 22781 external_mini_cluster.cc:1428] Setting key ab725045a36f2de34639468d4b4d7434
I20260806 01:13:01.085748 24130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4: Bootstrap starting.
I20260806 01:13:01.089275 24130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:01.090608 24130 log.cc:824] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:01.093830 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:01.081213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56303 (local address 127.22.63.126:39883)
0806 01:13:01.081987 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:01.081998 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:01.082044 (+ 46us) server_negotiation.cc:407] Connection header received
0806 01:13:01.084301 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:01.084324 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:01.084698 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:01.085001 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:01.085611 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:01.087004 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:01.088888 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:01.089613 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:01.089756 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:01.090982 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:01.090989 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:01.091002 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:13:01.091837 (+ 835us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:01.091938 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:13:01.091975 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:01.092103 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:01.092333 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 01:13:01.092605 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":358,"threads_started":1}
I20260806 01:13:01.094149 24130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4: No bootstrap required, opened a new log
I20260806 01:13:01.097645 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:13:01.102376 24130 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } }
I20260806 01:13:01.102748 24130 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:01.102872 24130 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8925586ae748d79356111924aeecd4, State: Initialized, Role: FOLLOWER
I20260806 01:13:01.103570 24130 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } }
I20260806 01:13:01.103909 24130 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:01.104070 24130 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:01.104274 24130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:01.107393 24130 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } }
I20260806 01:13:01.108043 24130 leader_election.cc:302] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff8925586ae748d79356111924aeecd4; no voters:
I20260806 01:13:01.108762 24130 leader_election.cc:288] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:01.108886 24147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:01.109789 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:01.102125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34747 (local address 127.22.63.126:39883)
0806 01:13:01.102389 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:01.102398 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:01.103314 (+ 916us) server_negotiation.cc:407] Connection header received
0806 01:13:01.103509 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:01.103513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:01.103578 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:01.103659 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:01.104334 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:01.105117 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:01.106958 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:01.107397 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:01.107499 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:01.108800 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:01.108806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:01.108810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:13:01.108833 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:01.108887 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:13:01.108909 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:01.109006 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:01.109197 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 01:13:01.109302 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:13:01.110956 24147 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [term 1 LEADER]: Becoming Leader. State: Replica: ff8925586ae748d79356111924aeecd4, State: Running, Role: LEADER
I20260806 01:13:01.111656 24130 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:01.111706 24147 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } }
I20260806 01:13:01.114774 24149 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff8925586ae748d79356111924aeecd4. Latest consensus state: current_term: 1 leader_uuid: "ff8925586ae748d79356111924aeecd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } } }
I20260806 01:13:01.114811 24148 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff8925586ae748d79356111924aeecd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8925586ae748d79356111924aeecd4" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39883 } } }
I20260806 01:13:01.115298 24149 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:01.115300 24148 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:01.121665 24162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:01.126694 24162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:01.136183 24162 catalog_manager.cc:1347] Generated new cluster ID: 1bc4ab18e654449684dd52fd8e63b3db
I20260806 01:13:01.136322 24162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:01.144817 24162 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:01.146704 24162 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:01.165841 24162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff8925586ae748d79356111924aeecd4: Generated new TSK 0
I20260806 01:13:01.167058 24162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:13:01.183166 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24040
W20260806 01:13:01.184448 24124 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56303
W20260806 01:13:01.185218 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:01.185384 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24040
2026-08-06T01:13:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2247 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:01 dist-test-slave-7f01 krb5kdc[24178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:01 dist-test-slave-7f01 krb5kdc[24178](info): set up 2 sockets
Aug 06 01:13:01 dist-test-slave-7f01 krb5kdc[24178](info): commencing operation
krb5kdc: starting...
W20260806 01:13:02.547964 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.004s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:13:02 dist-test-slave-7f01 krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:13:02.923668 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:39111
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:03.278453 24195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:03.278766 24195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:03.278860 24195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:03.286288 24195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:03.286422 24195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:03.286521 24195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:03.286649 24195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:03.286726 24195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:03.298012 24195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:03.299491 24195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:03.300940 24195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:03.313390 24202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:03.313488 24209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:03.313443 24207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:03.314801 24195 server_base.cc:1034] running on GCE node
I20260806 01:13:03.315588 24195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:03.317142 24195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:03.318382 24195 hybrid_clock.cc:648] HybridClock initialized: now 1785978783318337 us; error 40 us; skew 500 ppm
I20260806 01:13:03.318787 24195 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:03 dist-test-slave-7f01 krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978783, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:03.322221 24195 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:13:03.323930 24195 webserver.cc:460] Webserver started at http://127.22.63.126:33739/ using document root <none> and password file <none>
I20260806 01:13:03.324679 24195 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:03.324836 24195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:03.325165 24195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:03.328423 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "1cc702f0067545f6857c2124368d0e99"
format_stamp: "Formatted at 2026-08-06 01:13:03 on dist-test-slave-7f01"
server_key: "c35ef615ba88d8591d7f83ef8a0c423b"
server_key_iv: "bb409e68e94bdb1561e5173f0d536b4e"
server_key_version: "encryptionkey@0"
I20260806 01:13:03.329290 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "1cc702f0067545f6857c2124368d0e99"
format_stamp: "Formatted at 2026-08-06 01:13:03 on dist-test-slave-7f01"
server_key: "c35ef615ba88d8591d7f83ef8a0c423b"
server_key_iv: "bb409e68e94bdb1561e5173f0d536b4e"
server_key_version: "encryptionkey@0"
I20260806 01:13:03.334841 24195 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 01:13:03.338624 24220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:03.340240 24195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:03.340453 24195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "1cc702f0067545f6857c2124368d0e99"
format_stamp: "Formatted at 2026-08-06 01:13:03 on dist-test-slave-7f01"
server_key: "c35ef615ba88d8591d7f83ef8a0c423b"
server_key_iv: "bb409e68e94bdb1561e5173f0d536b4e"
server_key_version: "encryptionkey@0"
I20260806 01:13:03.340674 24195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:03.346951 24195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:03.351387 24195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:03.351724 24195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:03.372411 24195 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:39111
I20260806 01:13:03.372447 24297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:39111 every 8 connection(s)
I20260806 01:13:03.373903 24195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:03.377795 24301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:03.380151 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24195
I20260806 01:13:03.380363 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:13:03.380820 22781 external_mini_cluster.cc:1428] Setting key e974dc3f90a2f2733755a9c5a0266811
I20260806 01:13:03.389173 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:03.382717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47099 (local address 127.22.63.126:39111)
0806 01:13:03.383626 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:03.383637 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:03.383689 (+ 52us) server_negotiation.cc:407] Connection header received
0806 01:13:03.385429 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:03.385459 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:03.385817 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:03.386154 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:03.387417 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:03.387423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:03.387437 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:13:03.387466 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:03.387531 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:13:03.387552 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:03.387834 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:03.387947 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 01:13:03.388206 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":439,"threads_started":1}
I20260806 01:13:03.390573 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99: Bootstrap starting.
I20260806 01:13:03.393267 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:13:03.394407 24301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:03.395483 24301 log.cc:824] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:03.398787 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99: No bootstrap required, opened a new log
I20260806 01:13:03.400779 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:03.397968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51473 (local address 127.22.63.126:39111)
0806 01:13:03.398166 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:03.398173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:03.398594 (+ 421us) server_negotiation.cc:407] Connection header received
0806 01:13:03.398736 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:03.398744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:03.398819 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:03.398937 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:03.400020 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:03.400025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:03.400030 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 01:13:03.400048 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:03.400094 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 01:13:03.400115 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:03.400243 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:03.400397 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 01:13:03.400463 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 01:13:03.407200 24301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } }
I20260806 01:13:03.407701 24301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 01:13:03.407876 24301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc702f0067545f6857c2124368d0e99, State: Initialized, Role: FOLLOWER
I20260806 01:13:03.408473 24301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } }
I20260806 01:13:03.408687 24301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 01:13:03.408792 24301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 01:13:03.408917 24301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:03.411069 24301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } }
I20260806 01:13:03.411535 24301 leader_election.cc:302] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cc702f0067545f6857c2124368d0e99; no voters:
I20260806 01:13:03.412341 24301 leader_election.cc:288] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 01:13:03.412993 24315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:03.414294 24315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [term 1 LEADER]: Becoming Leader. State: Replica: 1cc702f0067545f6857c2124368d0e99, State: Running, Role: LEADER
I20260806 01:13:03.414860 24301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:03.414928 24315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } }
I20260806 01:13:03.418265 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cc702f0067545f6857c2124368d0e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } } }
I20260806 01:13:03.418320 24318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cc702f0067545f6857c2124368d0e99. Latest consensus state: current_term: 1 leader_uuid: "1cc702f0067545f6857c2124368d0e99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cc702f0067545f6857c2124368d0e99" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 39111 } } }
I20260806 01:13:03.418680 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:03.418722 24318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:03.420109 24328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:03.426404 24328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:03.435190 24328 catalog_manager.cc:1347] Generated new cluster ID: 55b51eb7460842fdaa2e305b95d325bc
I20260806 01:13:03.435382 24328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:03.458423 24328 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:03.459035 24328 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:03.473359 24328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cc702f0067545f6857c2124368d0e99: Generated new TSK 0
I20260806 01:13:03.474542 24328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:13:03.480201 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24195
W20260806 01:13:03.481343 24291 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47099
W20260806 01:13:03.482126 22781 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24195: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 01:13:03.482281 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24195
2026-08-06T01:13:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2300 ms)
[----------] 18 tests from AuthTokenIssuingTest (42056 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:03 dist-test-slave-7f01 krb5kdc[24348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:03 dist-test-slave-7f01 krb5kdc[24348](info): set up 2 sockets
Aug 06 01:13:03 dist-test-slave-7f01 krb5kdc[24348](info): commencing operation
krb5kdc: starting...
W20260806 01:13:04.852444 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.003s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:13:05 dist-test-slave-7f01 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:13:05.211668 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45925
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:05.557190 24368 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:05.557587 24368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:05.557717 24368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:05.557858 24368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:05.565001 24368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:05.565121 24368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:05.565248 24368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:05.565340 24368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:05.565445 24368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:05.575045 24368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:45925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:05.576380 24368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:05.577710 24368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:05.589756 24378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:05.589911 24374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:05.589962 24375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:05.590864 24368 server_base.cc:1034] running on GCE node
I20260806 01:13:05.591670 24368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:05.593019 24368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:05.594269 24368 hybrid_clock.cc:648] HybridClock initialized: now 1785978785594213 us; error 36 us; skew 500 ppm
I20260806 01:13:05.594607 24368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:05 dist-test-slave-7f01 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:05.597820 24368 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:13:05.599294 24368 webserver.cc:460] Webserver started at http://127.22.63.126:45041/ using document root <none> and password file <none>
I20260806 01:13:05.600041 24368 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:05.600175 24368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:05.600505 24368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:05.603410 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "6f7c6a02cf384d92b3611d7c0ee84d14"
format_stamp: "Formatted at 2026-08-06 01:13:05 on dist-test-slave-7f01"
server_key: "ea40796db0ef0fa1c3b8eee91a5f686b"
server_key_iv: "1af96b971336f6b84fce946ebbdd0ac5"
server_key_version: "encryptionkey@0"
I20260806 01:13:05.604231 24368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "6f7c6a02cf384d92b3611d7c0ee84d14"
format_stamp: "Formatted at 2026-08-06 01:13:05 on dist-test-slave-7f01"
server_key: "ea40796db0ef0fa1c3b8eee91a5f686b"
server_key_iv: "1af96b971336f6b84fce946ebbdd0ac5"
server_key_version: "encryptionkey@0"
I20260806 01:13:05.609256 24368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:13:05.612658 24385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:05.614014 24368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:05.614207 24368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "6f7c6a02cf384d92b3611d7c0ee84d14"
format_stamp: "Formatted at 2026-08-06 01:13:05 on dist-test-slave-7f01"
server_key: "ea40796db0ef0fa1c3b8eee91a5f686b"
server_key_iv: "1af96b971336f6b84fce946ebbdd0ac5"
server_key_version: "encryptionkey@0"
I20260806 01:13:05.614437 24368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:05.655880 24368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:05.660431 24368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:05.660744 24368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:05.680363 24368 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:45925
I20260806 01:13:05.680383 24456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:45925 every 8 connection(s)
I20260806 01:13:05.681831 24368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:05.685612 24457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:05.686299 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24368
I20260806 01:13:05.686538 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:13:05.687212 22781 external_mini_cluster.cc:1428] Setting key c06a53479ac5258be992c4c330754241
I20260806 01:13:05.692240 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:05 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:05.714993 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.695289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.22.63.126:45925)
0806 01:13:05.696989 (+ 1700us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:05.696998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:05.697048 (+ 50us) server_negotiation.cc:407] Connection header received
0806 01:13:05.698539 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:05.698545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:05.698752 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:05.698963 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:05.700282 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:05.701472 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:05.703566 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:05.704151 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:05.704349 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:05.707707 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:05.707715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:05.707727 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:05.707799 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:05.710575 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:05.711300 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:05.711307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:05.711309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:05.711445 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:05.711926 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:05.711929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:05.711931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:05.712431 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:05.712643 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:05.713173 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 01:13:05.713478 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":1476,"thread_start_us":1246,"threads_started":1}
I20260806 01:13:05.715026 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.695288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45925 (local address 127.0.0.1:60390)
0806 01:13:05.696176 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.696231 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:05.698176 (+ 1945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:05.698978 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:05.698984 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:05.699684 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:05.700072 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:05.700088 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:05.701634 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:05.701643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:05.703429 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:05.703442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:05.704397 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:05.704404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:05.704669 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:05.705334 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:05.705365 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:05.707478 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:05.710705 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:05.710719 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:05.710729 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:05.711149 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:05.711567 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:05.711572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:05.711575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:05.711731 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:05.712565 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:05.712577 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:05.712799 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 01:13:05.713037 (+ 238us) client_negotiation.cc:240] Negotiation successful
0806 01:13:05.713597 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":200,"thread_start_us":395,"threads_started":1}
I20260806 01:13:05.718429 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:05.719661 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.719257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33938)
0806 01:13:05.719428 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.719551 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:13:05.719972 24388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.63.125@KRBTEST.COM; defaulting to no policy
I20260806 01:13:05.721560 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.721220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33946)
0806 01:13:05.721373 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.721440 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:05.721779 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:38615: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
I20260806 01:13:05.762568 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:38615
--webserver_interface=127.22.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:05.771112 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } attempt: 1
I20260806 01:13:05.772413 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.771924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33962)
0806 01:13:05.772140 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.772253 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:13:05.773499 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.773185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33974)
0806 01:13:05.773313 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.773377 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:13:05.773770 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:38615: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
I20260806 01:13:05.842137 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } attempt: 2
I20260806 01:13:05.843508 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.842905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33982)
0806 01:13:05.843191 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.843310 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 01:13:05.844666 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.844251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33988)
0806 01:13:05.844409 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.844510 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:05.844873 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:38615: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
I20260806 01:13:05.936159 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } attempt: 3
I20260806 01:13:05.937551 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.936996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33990)
0806 01:13:05.937251 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.937377 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 01:13:05.938670 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:05.938249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:33994)
0806 01:13:05.938414 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:05.938497 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 01:13:05.938868 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:38615: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
I20260806 01:13:06.082159 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } attempt: 4
I20260806 01:13:06.083681 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.082942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:34010)
0806 01:13:06.083348 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.083479 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260806 01:13:06.084918 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.084466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:34026)
0806 01:13:06.084645 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.084754 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:13:06.085175 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:38615: Network error: Client connection negotiation failed: client connection to 127.22.63.125:38615: connect: Connection refused (error 111)
W20260806 01:13:06.116187 24472 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:06.116487 24472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:06.116575 24472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:06.116657 24472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:06.123939 24472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:06.124027 24472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:06.124084 24472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:06.124135 24472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:06.124188 24472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:06.133728 24472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:06.134949 24472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:06.136283 24472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:06.148275 24483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:06.149248 24482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:06.149243 24485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:06.150179 24472 server_base.cc:1034] running on GCE node
I20260806 01:13:06.150947 24472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:06.152392 24472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:06.153622 24472 hybrid_clock.cc:648] HybridClock initialized: now 1785978786153567 us; error 25 us; skew 500 ppm
I20260806 01:13:06.153934 24472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:06.157394 24472 init.cc:377] Logged in from keytab as kudu/127.22.63.125@KRBTEST.COM (short username kudu)
I20260806 01:13:06.158937 24472 webserver.cc:460] Webserver started at http://127.22.63.125:39975/ using document root <none> and password file <none>
I20260806 01:13:06.159641 24472 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:06.159780 24472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:06.160138 24472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:06.163060 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data/instance:
uuid: "71c6f1c12d7f488997823e3af4de263d"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "eb57eb18e4655191d5f6b87570b21bb5"
server_key_iv: "1631d4a570437b42b0520de00b570667"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.163930 24472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal/instance:
uuid: "71c6f1c12d7f488997823e3af4de263d"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "eb57eb18e4655191d5f6b87570b21bb5"
server_key_iv: "1631d4a570437b42b0520de00b570667"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.168875 24472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 01:13:06.172174 24494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:06.173537 24472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 01:13:06.173743 24472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "71c6f1c12d7f488997823e3af4de263d"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "eb57eb18e4655191d5f6b87570b21bb5"
server_key_iv: "1631d4a570437b42b0520de00b570667"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.173983 24472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:06.193122 24472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:06.197258 24472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:06.197554 24472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:06.216693 24472 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.125:38615
I20260806 01:13:06.216778 24569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.125:38615 every 8 connection(s)
I20260806 01:13:06.218116 24472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
I20260806 01:13:06.221818 24572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:06.227226 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24472
I20260806 01:13:06.227440 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-1/wal/instance
I20260806 01:13:06.227948 22781 external_mini_cluster.cc:1428] Setting key c17dc132ce4f7bbbffdc925f5a98319f
I20260806 01:13:06.228384 24572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:06.248643 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.230926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60392 (local address 127.22.63.126:45925)
0806 01:13:06.231415 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.231420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.232433 (+ 1013us) server_negotiation.cc:407] Connection header received
0806 01:13:06.234297 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.234303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.234398 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.234521 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.236069 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.236786 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.238278 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.238701 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.238836 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.242500 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.242510 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.242513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.242555 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.245381 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.246093 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.246105 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.246108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.246281 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.246829 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.246833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.246835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.247112 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.247234 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.248207 (+ 973us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.248374 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":273,"threads_started":1}
I20260806 01:13:06.250186 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.231502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45925 (local address 127.0.0.1:60392)
0806 01:13:06.232257 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.232295 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.234024 (+ 1729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.234755 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.234763 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.235254 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.235859 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.235877 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.236964 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.236976 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.238127 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.238139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.238942 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.238950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.239184 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.239907 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.239929 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.242260 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.245524 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.245533 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.245548 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.245964 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.246445 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.246450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.246453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.246703 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.247247 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.247257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.247711 (+ 454us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.248118 (+ 407us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.248652 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":339,"threads_started":1}
I20260806 01:13:06.252056 24572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.124@KRBTEST.COM" created.
I20260806 01:13:06.268353 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.252817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:34040)
0806 01:13:06.253060 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.253085 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.253212 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.254192 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.254200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.254733 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.254981 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.254997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.256540 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.256548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.258087 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.258104 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.258924 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.258940 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.259197 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.259836 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.259863 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.262441 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.265848 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.265851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.265854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.266207 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.266662 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.266665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.266667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.266797 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.267597 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.267601 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.267717 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.267901 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.268095 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 01:13:06.268889 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.253089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.22.63.125:38615)
0806 01:13:06.253772 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.253778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.253802 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:13:06.253842 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.253854 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.254020 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.254128 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.255186 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.256384 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.258235 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.258753 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.258850 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.262647 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.262658 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.262672 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.262705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.265728 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.266337 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.266343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.266345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.266528 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.266961 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.266968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.266971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.267443 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.267690 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.268098 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.268377 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":403,"threads_started":1}
I20260806 01:13:06.271175 24572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:06.272300 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.271886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40420)
0806 01:13:06.272040 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.272161 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:06.272562 24498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.274140 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.273801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40432)
0806 01:13:06.273976 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.274046 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:13:06.274374 24572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
I20260806 01:13:06.306735 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:41225
--webserver_interface=127.22.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:44215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:06.323666 24572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 1
I20260806 01:13:06.325050 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.324401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40448)
0806 01:13:06.324668 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.324814 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 01:13:06.326233 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.325775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40464)
0806 01:13:06.325975 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.326079 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:06.326470 24572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.384460 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } attempt: 5
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:06.394717 24572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 2
I20260806 01:13:06.395923 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.395337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40480)
0806 01:13:06.395544 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.395675 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 01:13:06.397059 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.396658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40494)
0806 01:13:06.396832 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.396928 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 01:13:06.397369 24572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.400563 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.385297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:34056)
0806 01:13:06.385512 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.385539 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.385671 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.386146 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.386150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.386541 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.386750 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.386759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.388244 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.388249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.389101 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.389113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.390136 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.390141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.390336 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.391069 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.391090 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.393566 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.397916 (+ 4350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.397921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.397924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.398332 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.398758 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.398762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.398764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.398908 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.399636 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.399640 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.399862 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.400004 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.400252 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:13:06.400878 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.385401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.22.63.125:38615)
0806 01:13:06.385618 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.385627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.385657 (+ 30us) server_negotiation.cc:407] Connection header received
0806 01:13:06.385807 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.385813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.385934 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.386079 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.386937 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.388076 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.389258 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.389950 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.390086 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.393785 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.393791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.393796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.393839 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.397748 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.398473 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.398479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.398482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.398640 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.399032 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.399039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.399045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.399490 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.399676 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.400171 (+ 495us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.400475 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:13:06.401934 24457 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:06.403040 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.402566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40510)
0806 01:13:06.402758 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.402852 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 01:13:06.404096 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.403691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40520)
0806 01:13:06.403867 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.403961 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 01:13:06.404322 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.455580 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 1
I20260806 01:13:06.457149 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.456477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40524)
0806 01:13:06.456746 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.456895 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 01:13:06.458238 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.457854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40528)
0806 01:13:06.458031 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.458116 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:13:06.458498 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.488639 24572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 3
I20260806 01:13:06.489960 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.489409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40542)
0806 01:13:06.489674 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.489783 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 01:13:06.491158 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.490672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40558)
0806 01:13:06.490938 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.491021 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260806 01:13:06.491398 24572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.526746 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 2
I20260806 01:13:06.528195 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.527507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40570)
0806 01:13:06.527811 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.527953 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 01:13:06.529320 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.528947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40576)
0806 01:13:06.529106 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.529193 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:13:06.529629 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.634624 24572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 4
I20260806 01:13:06.635936 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.635349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40582)
0806 01:13:06.635611 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.635695 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 01:13:06.635878 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 3
I20260806 01:13:06.637024 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.636547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40598)
0806 01:13:06.636764 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.636875 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:13:06.637140 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.636691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40600)
0806 01:13:06.636905 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.637000 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 01:13:06.637392 24572 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
I20260806 01:13:06.638748 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.637671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40602)
0806 01:13:06.638364 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.638527 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":403,"threads_started":1}
W20260806 01:13:06.638963 24457 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:41225: Network error: Client connection negotiation failed: client connection to 127.22.63.124:41225: connect: Connection refused (error 111)
W20260806 01:13:06.672639 24581 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:06.673009 24581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:06.673135 24581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:06.673273 24581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:06.680863 24581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:06.680989 24581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:06.681115 24581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:06.681202 24581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:06.681313 24581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:06.691179 24581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:44215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:45925,127.22.63.125:38615,127.22.63.124:41225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:41225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:06.692481 24581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:06.693822 24581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:06.705749 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:06.705703 24589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:06.705781 24593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:06.706897 24581 server_base.cc:1034] running on GCE node
I20260806 01:13:06.707696 24581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:06.709082 24581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:06.710340 24581 hybrid_clock.cc:648] HybridClock initialized: now 1785978786710253 us; error 50 us; skew 500 ppm
I20260806 01:13:06.710721 24581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:06.714005 24581 init.cc:377] Logged in from keytab as kudu/127.22.63.124@KRBTEST.COM (short username kudu)
I20260806 01:13:06.715550 24581 webserver.cc:460] Webserver started at http://127.22.63.124:46713/ using document root <none> and password file <none>
I20260806 01:13:06.716280 24581 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:06.716372 24581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:06.716642 24581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:06.719677 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data/instance:
uuid: "89f4f88e52934f328a5fc8a82b337f4c"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "8a4609b40c632997edefb9178e977316"
server_key_iv: "1ffb15ced91c60a328e3520d4efb66c6"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.720577 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal/instance:
uuid: "89f4f88e52934f328a5fc8a82b337f4c"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "8a4609b40c632997edefb9178e977316"
server_key_iv: "1ffb15ced91c60a328e3520d4efb66c6"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.725780 24581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 01:13:06.729224 24604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:06.730559 24581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 01:13:06.730698 24581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "89f4f88e52934f328a5fc8a82b337f4c"
format_stamp: "Formatted at 2026-08-06 01:13:06 on dist-test-slave-7f01"
server_key: "8a4609b40c632997edefb9178e977316"
server_key_iv: "1ffb15ced91c60a328e3520d4efb66c6"
server_key_version: "encryptionkey@0"
I20260806 01:13:06.730882 24581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:06.745321 24581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:06.749523 24581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:06.749742 24581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:06.769609 24581 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.124:41225
I20260806 01:13:06.769639 24674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.124:41225 every 8 connection(s)
I20260806 01:13:06.771036 24581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
I20260806 01:13:06.773470 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24581
I20260806 01:13:06.773669 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785978402713339-22781-0/minicluster-data/master-2/wal/instance
I20260806 01:13:06.774204 22781 external_mini_cluster.cc:1428] Setting key a06c239e264903bdc7c5933da4bd593c
I20260806 01:13:06.775400 24676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:06.785060 24676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:06.799316 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.776169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.22.63.124:41225)
0806 01:13:06.776983 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.776995 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.777038 (+ 43us) server_negotiation.cc:407] Connection header received
0806 01:13:06.778695 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.778715 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.779110 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.779448 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.781371 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.782742 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.784594 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.785279 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.785410 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.790659 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.790670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.790684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.790733 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.794744 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.795618 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.795628 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.795638 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.795885 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.796284 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.796292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.796297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.796891 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.797167 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.797594 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.798049 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":365,"threads_started":1}
I20260806 01:13:06.805580 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.788282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:45925 (local address 127.0.0.1:60398)
0806 01:13:06.789044 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.789094 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.789237 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.790029 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.790037 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.790842 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.791227 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.791241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.792634 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.792641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.794520 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.794543 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.795568 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.795577 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.795815 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.796503 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.796522 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.799039 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.802566 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.802579 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.802588 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.803121 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.803549 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.803557 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.803560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.803713 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.804452 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.804460 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.804646 (+ 186us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.804822 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.805144 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":333,"threads_started":1}
I20260806 01:13:06.805601 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.788393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.22.63.126:45925)
0806 01:13:06.789651 (+ 1258us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.789659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.789681 (+ 22us) server_negotiation.cc:407] Connection header received
0806 01:13:06.789746 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.789750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.789821 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.789975 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.791469 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.792411 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.794684 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.795352 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.795489 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.799250 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.799259 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.799265 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.799314 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.802401 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.803259 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.803265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.803269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.803424 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.803894 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.803899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.803902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.804316 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.804497 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.804948 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.805175 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":1009,"threads_started":1}
I20260806 01:13:06.807185 24676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:06.816176 24457 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 4
I20260806 01:13:06.820293 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.802492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60412 (local address 127.22.63.126:45925)
0806 01:13:06.803121 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.803125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.803143 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:13:06.803181 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.803184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.803252 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.803378 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.804816 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.805755 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.807362 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.807982 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.808112 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.813061 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.813068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.813074 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.813114 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.816232 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.817126 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.817134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.817139 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.817315 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.817790 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.817797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.817802 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.818245 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.818520 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.819576 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.819812 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":336,"threads_started":1}
I20260806 01:13:06.824122 24577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.808072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.22.63.125:38615)
0806 01:13:06.808241 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.808246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.808449 (+ 203us) server_negotiation.cc:407] Connection header received
0806 01:13:06.808606 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.808611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.808710 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.808811 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.809746 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.810414 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.812013 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.812756 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.812910 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.816564 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.816571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.816577 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.816619 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.820188 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.820911 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.820917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.820921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.821089 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.821677 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.821683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.821688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.822029 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.822204 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.823392 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.823635 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:13:06.824599 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.808073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:38615 (local address 127.0.0.1:34060)
0806 01:13:06.808345 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.808359 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.808491 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.808882 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.808887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.809370 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.809585 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.809599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.810566 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.810573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.811838 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.811852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.812933 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.812939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.813129 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.813669 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.813697 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.816288 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.820314 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.820320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.820324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.820778 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.821240 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.821245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.821249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.821429 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.822168 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.822173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.822428 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.822599 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.822847 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1,"spinlock_wait_cycles":31616}
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:06.825701 24676 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:06.834403 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.817048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.22.63.124:41225)
0806 01:13:06.817217 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.817223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.817251 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:13:06.817322 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.817328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.817402 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.817500 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.818558 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.819668 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.821622 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.822111 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.822227 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.826308 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.826314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.826318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.826357 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.830417 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.831312 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.831318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.831322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.831502 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.831991 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.831999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.832005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.832366 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.832541 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.833634 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.833910 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65,"spinlock_wait_cycles":24320}
I20260806 01:13:06.835135 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.816771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40622)
0806 01:13:06.816945 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.816965 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.817075 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.817521 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.817528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.818054 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.818394 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.818408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.819905 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.819911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.821456 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.821474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.822524 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.822533 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.822735 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.823361 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.823379 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.826120 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.830582 (+ 4462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.830591 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.830596 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.831151 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.831639 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.831647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.831652 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.831866 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.832532 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.832537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.832739 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.832908 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.834826 (+ 1918us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:06.841759 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: Bootstrap starting.
I20260806 01:13:06.842562 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.822714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.22.63.125:38615)
0806 01:13:06.823435 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.823441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.823462 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:13:06.823515 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.823519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.823588 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.823697 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.825520 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.826688 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.828604 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.829291 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.829422 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.834666 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.834676 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.834683 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.834732 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.838647 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.839500 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.839506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.839510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.839677 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.840104 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.840109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.840113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.840456 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.840745 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.841881 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.842134 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":332,"threads_started":1}
I20260806 01:13:06.842586 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.826369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40630)
0806 01:13:06.826627 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.826654 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.826753 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.828298 (+ 1545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.828303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.828859 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.829063 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.829078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.830747 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.830754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.831916 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.831929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.832872 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.832878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.833054 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.833566 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.833584 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.836081 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.839713 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.839719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.839723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.840247 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.840680 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.840684 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.840687 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.840856 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.841546 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.841550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.841706 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.841893 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.842215 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260806 01:13:06.843209 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.826677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40630 (local address 127.22.63.124:41225)
0806 01:13:06.827840 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.827845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.827873 (+ 28us) server_negotiation.cc:407] Connection header received
0806 01:13:06.827919 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.827924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.828099 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.828236 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.829256 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.830632 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.832092 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.832715 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.832855 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.836329 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.836336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.836341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.836377 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.839576 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.840381 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.840394 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.840397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.840553 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.841000 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.841006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.841009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.841401 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.841553 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.842095 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.842352 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":999,"thread_start_us":797,"threads_started":1}
I20260806 01:13:06.846619 24457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:06.848048 24457 log.cc:824] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:06.848166 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:13:06.848541 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: Bootstrap starting.
I20260806 01:13:06.851589 24457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: No bootstrap required, opened a new log
I20260806 01:13:06.852041 24676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:06.853360 24676 log.cc:824] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:06.851536 24419 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60412
I20260806 01:13:06.856648 24676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: No bootstrap required, opened a new log
I20260806 01:13:06.859586 24457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:06.860004 24457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f7c6a02cf384d92b3611d7c0ee84d14, State: Initialized, Role: FOLLOWER
I20260806 01:13:06.860436 24457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:06.861374 24690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:06.861789 24690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:06.862607 24457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:06.862703 24419 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:13:06.862914 24419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:06.866865 24419 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:06.867702 24676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:06.868238 24676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f4f88e52934f328a5fc8a82b337f4c, State: Initialized, Role: FOLLOWER
I20260806 01:13:06.868825 24676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:06.869684 24419 leader_election.cc:288] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [CANDIDATE]: Term 1 election: Requested vote from peers 71c6f1c12d7f488997823e3af4de263d (127.22.63.125:38615), 89f4f88e52934f328a5fc8a82b337f4c (127.22.63.124:41225)
I20260806 01:13:06.874681 24676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:06.875859 24644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89f4f88e52934f328a5fc8a82b337f4c"
I20260806 01:13:06.881920 24644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:06.872520 24711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
W20260806 01:13:06.884356 24709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:13:06.884558 24709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:13:06.886734 24644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7c6a02cf384d92b3611d7c0ee84d14 in term 1.
I20260806 01:13:06.896104 24711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:06.896965 24388 leader_election.cc:302] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f7c6a02cf384d92b3611d7c0ee84d14, 89f4f88e52934f328a5fc8a82b337f4c; no voters:
I20260806 01:13:06.897683 24690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 01:13:06.874189 24368 master_runner.cc:186] Error getting master registration for 127.22.63.125:38615: OK, instance_id { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" instance_seqno: 1785978786211147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:13:06.904162 24368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:13:06.910218 24690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [term 1 LEADER]: Becoming Leader. State: Replica: 6f7c6a02cf384d92b3611d7c0ee84d14, State: Running, Role: LEADER
I20260806 01:13:06.912055 24690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
W20260806 01:13:06.914013 24393 tablet.cc:2369] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:06.924994 24725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7c6a02cf384d92b3611d7c0ee84d14. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:06.925563 24725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:06.928469 24741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:06.936621 24572 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } attempt: 5
I20260806 01:13:06.943953 24741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 01:13:06 dist-test-slave-7f01 krb5kdc[24348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:06.963078 24575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.937418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:41225 (local address 127.0.0.1:40648)
0806 01:13:06.937922 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:06.937947 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:06.938072 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:06.939176 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:06.939182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:06.939931 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:06.940210 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.940225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.942074 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.942083 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.944655 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:06.944677 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.946288 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.946294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:06.946561 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.947389 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:06.947423 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:06.951304 (+ 3881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:06.958539 (+ 7235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.958550 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.958558 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.959423 (+ 865us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.960130 (+ 707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:06.960139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:06.960146 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:06.960401 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:06.961515 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:06.961523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:06.961771 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 01:13:06.961974 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 01:13:06.962458 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329}
I20260806 01:13:06.964903 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.928106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.22.63.124:41225)
0806 01:13:06.928326 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.928333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.934647 (+ 6314us) server_negotiation.cc:407] Connection header received
0806 01:13:06.934837 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.934842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.934925 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.935027 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.938847 (+ 3820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.940011 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.943077 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.943840 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.943948 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.949865 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.949875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.949884 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.949938 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.954224 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.959957 (+ 5733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.959966 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.959973 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.960163 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.961154 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.961188 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.961194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.961689 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.961989 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.963330 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.963966 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:13:06.965574 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.938332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.22.63.124:41225)
0806 01:13:06.938589 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.938598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.938621 (+ 23us) server_negotiation.cc:407] Connection header received
0806 01:13:06.938697 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.938705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.938810 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.938939 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.940427 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.941863 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.944890 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.945844 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.945993 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.951592 (+ 5599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.951602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.951610 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.951663 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.958337 (+ 6674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.959647 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.959657 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.959665 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.959937 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.960581 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.960589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.960596 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.961148 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:06.961422 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:06.964652 (+ 3230us) server_negotiation.cc:299] Negotiation successful
0806 01:13:06.964979 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":78}
I20260806 01:13:06.982043 24644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [term 1 FOLLOWER]: Refusing update from remote peer 6f7c6a02cf384d92b3611d7c0ee84d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:06.983047 24690 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:06.991075 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: Bootstrap starting.
W20260806 01:13:07.001627 24612 tablet.cc:2369] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:07.003486 24572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:07.008188 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.913421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.22.63.126:45925)
0806 01:13:06.913739 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.913747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.927463 (+ 13716us) server_negotiation.cc:407] Connection header received
0806 01:13:06.930348 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.930354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.930443 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.930520 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.934843 (+ 4323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.936337 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.966068 (+ 29731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.967190 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.967360 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.986428 (+ 19068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.986440 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.986450 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.986515 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.993285 (+ 6770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.997302 (+ 4017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.997315 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.997321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.997762 (+ 441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:07.000151 (+ 2389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:07.000197 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:07.000203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:07.001427 (+ 1224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:07.002290 (+ 863us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.006104 (+ 3814us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.006908 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 01:13:07.009073 24572 log.cc:824] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:07.010495 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:06.913491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.22.63.125:38615)
0806 01:13:06.914953 (+ 1462us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:06.914960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:06.929094 (+ 14134us) server_negotiation.cc:407] Connection header received
0806 01:13:06.929922 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:06.929930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:06.930025 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:06.930105 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:06.933304 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.934621 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.968847 (+ 34226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:06.969909 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:06.970049 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:06.978810 (+ 8761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:06.978819 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:06.978824 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:06.978876 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:06.985904 (+ 7028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:06.998143 (+ 12239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:06.998153 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:06.998160 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:06.998386 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:07.002627 (+ 4241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:07.002636 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:07.002643 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:07.003180 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:07.003533 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.009432 (+ 5899us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.009927 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289}
I20260806 01:13:07.027423 24572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: No bootstrap required, opened a new log
I20260806 01:13:07.038597 24711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7c6a02cf384d92b3611d7c0ee84d14. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.038949 24711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:07.041254 24748 mvcc.cc:205] Tried to move back new op lower bound from 7315369111459508224 to 7315369111205474304. Current Snapshot: MvccSnapshot[applied={T|T < 7315369111459508224}]
I20260806 01:13:07.043336 24572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:07.043812 24572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71c6f1c12d7f488997823e3af4de263d, State: Initialized, Role: FOLLOWER
I20260806 01:13:07.044387 24572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } }
I20260806 01:13:07.026721 24581 master_runner.cc:186] Error getting master registration for 127.22.63.125:38615: OK, instance_id { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" instance_seqno: 1785978786211147 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:13:07.045095 24581 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:13:07.045789 24690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.047531 24690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:07.047713 24711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.048360 24711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:07.050216 24741 catalog_manager.cc:1347] Generated new cluster ID: 8d6aa213d41749bba24e49758433bb48
I20260806 01:13:07.050334 24741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:07.068749 24572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:07.068778 24533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "71c6f1c12d7f488997823e3af4de263d"
I20260806 01:13:07.069306 24532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:07.071514 24741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:07.072991 24532 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [term 1 FOLLOWER]: Refusing update from remote peer 6f7c6a02cf384d92b3611d7c0ee84d14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:07.073518 24741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:07.076972 24533 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f7c6a02cf384d92b3611d7c0ee84d14 in term 1.
I20260806 01:13:07.082428 24725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:07.067293 24757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.092358 24757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: This master's current role is: FOLLOWER
W20260806 01:13:07.096174 24749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:13:07.096318 24749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:13:07.111013 24741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14: Generated new TSK 0
I20260806 01:13:07.112116 24741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:13:07.135329 24767 mvcc.cc:205] Tried to move back new op lower bound from 7315369111459508224 to 7315369111205474304. Current Snapshot: MvccSnapshot[applied={T|T < 7315369111459508224}]
I20260806 01:13:07.141011 24757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f7c6a02cf384d92b3611d7c0ee84d14. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.141390 24757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:07.141541 24690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.141863 24690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f7c6a02cf384d92b3611d7c0ee84d14 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:07.141758 24764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f7c6a02cf384d92b3611d7c0ee84d14" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 45925 } } peers { permanent_uuid: "71c6f1c12d7f488997823e3af4de263d" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 38615 } } peers { permanent_uuid: "89f4f88e52934f328a5fc8a82b337f4c" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 41225 } } }
I20260806 01:13:07.142097 24764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:07.279528 24784 catalog_manager.cc:1259] Loaded cluster ID: 8d6aa213d41749bba24e49758433bb48
I20260806 01:13:07.279664 24784 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: loading cluster ID for follower catalog manager: success
W20260806 01:13:07.281203 22781 mini_kdc.cc:348] Time spent kdestroy: real 0.154s user 0.000s sys 0.041s
I20260806 01:13:07.284051 24784 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: acquiring CA information for follower catalog manager: success
I20260806 01:13:07.286700 24784 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71c6f1c12d7f488997823e3af4de263d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 01:13:07.365985 24686 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.336528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.22.63.125:38615)
0806 01:13:07.336740 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.336747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.361794 (+ 25047us) server_negotiation.cc:407] Connection header received
0806 01:13:07.364064 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.364073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.364215 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.364308 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.365670 (+ 1362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33278: BlockingRecv error: recv got EOF from 127.0.0.1:33278 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260806 01:13:07.369431 24685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.336216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.22.63.126:45925)
0806 01:13:07.336557 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.336565 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.367146 (+ 30581us) server_negotiation.cc:407] Connection header received
0806 01:13:07.367657 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.367663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.367790 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.367880 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.369108 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46860: BlockingRecv error: recv got EOF from 127.0.0.1:46860 (error 108)
Metrics: {"server-negotiator.queue_time_us":186}
W20260806 01:13:07.372983 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.362650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45784 (local address 127.22.63.124:41225)
0806 01:13:07.362873 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.362880 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.370678 (+ 7798us) server_negotiation.cc:407] Connection header received
0806 01:13:07.370960 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.370967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.371111 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.371199 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.372709 (+ 1510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45784: BlockingRecv error: recv got EOF from 127.0.0.1:45784 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:13:07.437323 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.416709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.22.63.125:38615)
0806 01:13:07.416960 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.416971 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.428371 (+ 11400us) server_negotiation.cc:407] Connection header received
0806 01:13:07.428706 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.428732 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.428992 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.429162 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.429982 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.431272 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.433643 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.434477 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.434596 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.435528 (+ 932us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.436354 (+ 826us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.436496 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.436816 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.436917 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 01:13:07.485157 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.474766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.22.63.124:41225)
0806 01:13:07.474976 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.474983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.482635 (+ 7652us) server_negotiation.cc:407] Connection header received
0806 01:13:07.483286 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.483294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.483470 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.483561 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.484821 (+ 1260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45800: BlockingRecv error: recv got EOF from 127.0.0.1:45800 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 01:13:07.532879 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.516748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.22.63.125:38615)
0806 01:13:07.516967 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.516974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.524990 (+ 8016us) server_negotiation.cc:407] Connection header received
0806 01:13:07.525533 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.525541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.525641 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.525721 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.526478 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.527840 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.530232 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.531040 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.531148 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.531938 (+ 790us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.532098 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.532169 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.532444 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.532541 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:13:07.576462 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.566542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.22.63.124:41225)
0806 01:13:07.566789 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.566796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.574243 (+ 7447us) server_negotiation.cc:407] Connection header received
0806 01:13:07.574808 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.574815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.574924 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.575000 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.576188 (+ 1188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45814: BlockingRecv error: recv got EOF from 127.0.0.1:45814 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 01:13:07.588176 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.582478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.22.63.125:38615)
0806 01:13:07.582746 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.582761 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.583293 (+ 532us) server_negotiation.cc:407] Connection header received
0806 01:13:07.583523 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.583528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.583600 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.583703 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.584190 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.584972 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.586340 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.586784 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.586929 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.587348 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.587476 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.587588 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.587758 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.587843 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260806 01:13:07.597227 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.594832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.22.63.124:41225)
0806 01:13:07.595089 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.595098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.595643 (+ 545us) server_negotiation.cc:407] Connection header received
0806 01:13:07.595940 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.595944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.596020 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.596131 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.596949 (+ 818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45828: BlockingRecv error: recv got EOF from 127.0.0.1:45828 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 01:13:07.610329 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.604480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33312 (local address 127.22.63.125:38615)
0806 01:13:07.604710 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.604720 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.605259 (+ 539us) server_negotiation.cc:407] Connection header received
0806 01:13:07.605461 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.605473 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.605541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.605637 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.606060 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.606865 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.608415 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.608886 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.609035 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.609523 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.609648 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.609749 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.609899 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.609991 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 01:13:07.618453 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.616788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.22.63.124:41225)
0806 01:13:07.617035 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.617041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.617062 (+ 21us) server_negotiation.cc:407] Connection header received
0806 01:13:07.617143 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.617149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.617234 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.617355 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.618205 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45830: BlockingRecv error: recv got EOF from 127.0.0.1:45830 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
I20260806 01:13:07.635334 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.629355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.22.63.125:38615)
0806 01:13:07.629604 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.629617 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.630189 (+ 572us) server_negotiation.cc:407] Connection header received
0806 01:13:07.630410 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.630415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.630510 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.630619 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.631065 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.631901 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.633230 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.633708 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.633876 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.634353 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.634454 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.634565 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.634774 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.634867 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 01:13:07.653800 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.649089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45842 (local address 127.22.63.124:41225)
0806 01:13:07.649368 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.649377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.652328 (+ 2951us) server_negotiation.cc:407] Connection header received
0806 01:13:07.652546 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.652553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.652638 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.652752 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.653512 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45842: BlockingRecv error: recv got EOF from 127.0.0.1:45842 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 01:13:07.679014 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.672891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.22.63.125:38615)
0806 01:13:07.673114 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.673123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.673686 (+ 563us) server_negotiation.cc:407] Connection header received
0806 01:13:07.673906 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.673910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.673988 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.674090 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.674526 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.675503 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.676981 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.677555 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.677691 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.678143 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.678272 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.678373 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.678601 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.678688 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:13:07.735555 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.728018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.22.63.124:41225)
0806 01:13:07.728252 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.728262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.733948 (+ 5686us) server_negotiation.cc:407] Connection header received
0806 01:13:07.734214 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.734238 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.734331 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.734449 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.735296 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45854: BlockingRecv error: recv got EOF from 127.0.0.1:45854 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:13:07.783330 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.776125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.22.63.125:38615)
0806 01:13:07.776355 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.776363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.777019 (+ 656us) server_negotiation.cc:407] Connection header received
0806 01:13:07.777233 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.777239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.777347 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.777450 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.777926 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.779206 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.781195 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.781849 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.781964 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.782486 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.782665 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.782745 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.782934 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.783017 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 01:13:07.801903 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.796721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.22.63.124:41225)
0806 01:13:07.796903 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.796909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.800415 (+ 3506us) server_negotiation.cc:407] Connection header received
0806 01:13:07.800637 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.800642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.800719 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.800834 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.801630 (+ 796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45870: BlockingRecv error: recv got EOF from 127.0.0.1:45870 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:13:07.876550 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:07.869997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.22.63.125:38615)
0806 01:13:07.870275 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.870284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.870605 (+ 321us) server_negotiation.cc:407] Connection header received
0806 01:13:07.870786 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.870793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.870886 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.870998 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:07.871419 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.872674 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.874242 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:07.874955 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:07.875096 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:07.875633 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:07.875820 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:07.875907 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:07.876101 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 01:13:07.876181 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:13:07.887822 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:07.885753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45874 (local address 127.22.63.124:41225)
0806 01:13:07.886072 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:07.886083 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:07.886109 (+ 26us) server_negotiation.cc:407] Connection header received
0806 01:13:07.886236 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:07.886242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:07.886370 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:07.886506 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:07.887540 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45874: BlockingRecv error: recv got EOF from 127.0.0.1:45874 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 01:13:08.030475 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.025548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33352 (local address 127.22.63.125:38615)
0806 01:13:08.025744 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.025748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.026126 (+ 378us) server_negotiation.cc:407] Connection header received
0806 01:13:08.026289 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.026293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.026366 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.026466 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.026868 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.027586 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.028982 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.029394 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.029487 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.029862 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.029973 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.030041 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.030181 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.030249 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 01:13:08.038568 24680 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:08.036836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.22.63.124:41225)
0806 01:13:08.037064 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.037070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.037334 (+ 264us) server_negotiation.cc:407] Connection header received
0806 01:13:08.037459 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.037463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.037538 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.037617 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:08.038361 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45886: BlockingRecv error: recv got EOF from 127.0.0.1:45886 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:13:08.098361 24749 catalog_manager.cc:1259] Loaded cluster ID: 8d6aa213d41749bba24e49758433bb48
I20260806 01:13:08.098470 24749 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: loading cluster ID for follower catalog manager: success
I20260806 01:13:08.100992 24749 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: acquiring CA information for follower catalog manager: success
I20260806 01:13:08.102712 24749 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89f4f88e52934f328a5fc8a82b337f4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:13:08.303305 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.298416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.22.63.125:38615)
0806 01:13:08.298624 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.298629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.298924 (+ 295us) server_negotiation.cc:407] Connection header received
0806 01:13:08.299095 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.299099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.299185 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.299271 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.299715 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.300427 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.301743 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.302121 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.302208 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.302587 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.302682 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.302747 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.302904 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.303013 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 01:13:08.314590 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.309069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.22.63.124:41225)
0806 01:13:08.309288 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.309294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.309445 (+ 151us) server_negotiation.cc:407] Connection header received
0806 01:13:08.309666 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.309670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.309824 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.309950 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.310401 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.311096 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.312451 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.312857 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.312990 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.313484 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.314019 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.314179 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.314238 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.314320 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 01:13:08.328433 24680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.322107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.22.63.124:41225)
0806 01:13:08.322285 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.322293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.322688 (+ 395us) server_negotiation.cc:407] Connection header received
0806 01:13:08.322888 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.322893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.322968 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.323064 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.323722 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.324750 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.326395 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.326824 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.326956 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.327579 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.327724 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.327832 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.328033 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.328106 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 01:13:08.329432 24686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.320698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.22.63.125:38615)
0806 01:13:08.320916 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.320926 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.322078 (+ 1152us) server_negotiation.cc:407] Connection header received
0806 01:13:08.322360 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.322368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.322466 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.322524 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.323423 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.324297 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.326441 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.327093 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.327224 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.327730 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.327901 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.328004 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.329032 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.329109 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 01:13:08.330060 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:08.320654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.22.63.126:45925)
0806 01:13:08.321693 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:08.321701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:08.321725 (+ 24us) server_negotiation.cc:407] Connection header received
0806 01:13:08.321911 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:08.321917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:08.322313 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:08.322509 (+ 196us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:08.323486 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.324642 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.327442 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:08.328035 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:08.328188 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:08.328697 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:08.329140 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:08.329259 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:08.329706 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 01:13:08.329794 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":900,"thread_start_us":783,"threads_started":1}
I20260806 01:13:08.335260 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24368
I20260806 01:13:08.336326 24452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:08.448297 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24368
I20260806 01:13:08.480222 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24472
I20260806 01:13:08.481420 24563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:08.589659 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24472
I20260806 01:13:08.618485 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24581
I20260806 01:13:08.619680 24670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:08.727353 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24581
2026-08-06T01:13:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5247 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 01:13:08 dist-test-slave-7f01 krb5kdc[24965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 01:13:08 dist-test-slave-7f01 krb5kdc[24965](info): set up 2 sockets
Aug 06 01:13:08 dist-test-slave-7f01 krb5kdc[24965](info): commencing operation
krb5kdc: starting...
W20260806 01:13:10.080214 22781 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.008s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/test-user.keytab.
Aug 06 01:13:10 dist-test-slave-7f01 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T01:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T01:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.63.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
Entry for principal kudu/127.22.63.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab.
I20260806 01:13:10.427256 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37241
--webserver_interface=127.22.63.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 01:13:10.776240 24988 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:10.776609 24988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:10.776736 24988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:10.776876 24988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:10.784008 24988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:10.784123 24988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:10.784252 24988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:10.784339 24988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:10.784451 24988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:10.794060 24988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.126.keytab
--principal=kudu/127.22.63.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.126:37241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.63.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:10.795425 24988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:10.796761 24988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:10.808732 24998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:10.808801 25000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:10.808823 25002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:10.809630 24988 server_base.cc:1034] running on GCE node
I20260806 01:13:10.810416 24988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:10.811723 24988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:10.813019 24988 hybrid_clock.cc:648] HybridClock initialized: now 1785978790812954 us; error 36 us; skew 500 ppm
I20260806 01:13:10.813359 24988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:10 dist-test-slave-7f01 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:10.821379 24988 init.cc:377] Logged in from keytab as kudu/127.22.63.126@KRBTEST.COM (short username kudu)
I20260806 01:13:10.822922 24988 webserver.cc:460] Webserver started at http://127.22.63.126:46425/ using document root <none> and password file <none>
I20260806 01:13:10.823633 24988 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:10.823796 24988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:10.824256 24988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:10.827200 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data/instance:
uuid: "a8eea8b8c93749b8b92576fb64692ef7"
format_stamp: "Formatted at 2026-08-06 01:13:10 on dist-test-slave-7f01"
server_key: "eb0c8e7c10764695facee618cb755cc2"
server_key_iv: "c089c6cd9c2ac641d0267b57f4610ea8"
server_key_version: "encryptionkey@0"
I20260806 01:13:10.828066 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance:
uuid: "a8eea8b8c93749b8b92576fb64692ef7"
format_stamp: "Formatted at 2026-08-06 01:13:10 on dist-test-slave-7f01"
server_key: "eb0c8e7c10764695facee618cb755cc2"
server_key_iv: "c089c6cd9c2ac641d0267b57f4610ea8"
server_key_version: "encryptionkey@0"
I20260806 01:13:10.833093 24988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 01:13:10.836524 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:10.837831 24988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:13:10.838025 24988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal
uuid: "a8eea8b8c93749b8b92576fb64692ef7"
format_stamp: "Formatted at 2026-08-06 01:13:10 on dist-test-slave-7f01"
server_key: "eb0c8e7c10764695facee618cb755cc2"
server_key_iv: "c089c6cd9c2ac641d0267b57f4610ea8"
server_key_version: "encryptionkey@0"
I20260806 01:13:10.838244 24988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:10.853909 24988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:10.858172 24988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:10.858456 24988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:10.877566 24988 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.126:37241
I20260806 01:13:10.877585 25093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.126:37241 every 8 connection(s)
I20260806 01:13:10.878968 24988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/data/info.pb
I20260806 01:13:10.879629 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 24988
I20260806 01:13:10.879906 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-0/wal/instance
I20260806 01:13:10.880481 22781 external_mini_cluster.cc:1428] Setting key c126a4563a5c6cbfd0e4cc32e15f76e8
I20260806 01:13:10.882844 25094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:10.889307 25094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:10 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:10.913687 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.892305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37241 (local address 127.0.0.1:58308)
0806 01:13:10.893466 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:10.893513 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:10.896046 (+ 2533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:10.897067 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:10.897072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:10.897627 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:10.897950 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:10.897967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:10.899534 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:10.899545 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:10.900645 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:10.900657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:10.901567 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:10.901574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:10.901762 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:10.902440 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:10.902463 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:10.904681 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:10.909346 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:10.909357 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:10.909368 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:10.909776 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:10.910294 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:10.910297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:10.910300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:10.910470 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:10.911578 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:10.911584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:10.911854 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 01:13:10.912107 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 01:13:10.912615 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"spinlock_wait_cycles":58624,"thread_start_us":655,"threads_started":1}
I20260806 01:13:10.913728 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.892306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58308 (local address 127.22.63.126:37241)
0806 01:13:10.893196 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:10.893210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:10.893653 (+ 443us) server_negotiation.cc:407] Connection header received
0806 01:13:10.896456 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:10.896468 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:10.896812 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:10.897009 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:10.898181 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:10.899371 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:10.900788 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:10.901328 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:10.901527 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:10.904942 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:10.904954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:10.904973 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:10.905018 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:10.909187 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:10.909938 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:10.909949 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:10.909955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:10.910153 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:10.910617 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:10.910625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:10.910631 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:10.911405 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:10.911623 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:10.912330 (+ 707us) server_negotiation.cc:299] Negotiation successful
0806 01:13:10.912834 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":608,"thread_start_us":354,"threads_started":1}
WARNING: no policy specified for kudu/127.22.63.125@KRBTEST.COM; defaulting to no policy
I20260806 01:13:10.917081 25094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:10.918171 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.917730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:57972)
0806 01:13:10.917941 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:10.918062 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260806 01:13:10.918442 25017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.125@KRBTEST.COM" created.
I20260806 01:13:10.920106 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.919671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:57982)
0806 01:13:10.919905 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:10.919974 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 01:13:10.920292 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:35827: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
Entry for principal kudu/127.22.63.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab.
I20260806 01:13:10.959990 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:35827
--webserver_interface=127.22.63.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:10.969661 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } attempt: 1
I20260806 01:13:10.971083 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.970449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:57988)
0806 01:13:10.970719 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:10.970882 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 01:13:10.972457 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:10.971978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58000)
0806 01:13:10.972166 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:10.972267 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 01:13:10.972718 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:35827: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
I20260806 01:13:11.041025 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } attempt: 2
I20260806 01:13:11.042399 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.041838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58008)
0806 01:13:11.042112 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.042221 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 01:13:11.043491 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.043122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58022)
0806 01:13:11.043285 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.043367 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:11.043669 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:35827: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
I20260806 01:13:11.135038 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } attempt: 3
I20260806 01:13:11.136307 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.135854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58034)
0806 01:13:11.136073 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.136167 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:13:11.137308 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.136952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58036)
0806 01:13:11.137114 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.137178 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 01:13:11.137485 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:35827: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
I20260806 01:13:11.280776 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } attempt: 4
I20260806 01:13:11.282104 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.281544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58044)
0806 01:13:11.281878 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.281965 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260806 01:13:11.283133 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.282769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58046)
0806 01:13:11.282931 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.283011 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:13:11.283321 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.125:35827: Network error: Client connection negotiation failed: client connection to 127.22.63.125:35827: connect: Connection refused (error 111)
W20260806 01:13:11.309696 25104 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:11.310093 25104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:11.310225 25104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:11.310361 25104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:11.317683 25104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:11.317816 25104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:11.317937 25104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:11.318027 25104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:11.318121 25104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:11.328370 25104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.125.keytab
--principal=kudu/127.22.63.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.125:35827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.63.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:11.329777 25104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:11.331205 25104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:11.343520 25113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:11.343575 25114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:11.343763 25119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:11.344470 25104 server_base.cc:1034] running on GCE node
I20260806 01:13:11.345340 25104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:11.346817 25104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:11.348083 25104 hybrid_clock.cc:648] HybridClock initialized: now 1785978791348052 us; error 72 us; skew 500 ppm
I20260806 01:13:11.348459 25104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:11 dist-test-slave-7f01 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:11.351720 25104 init.cc:377] Logged in from keytab as kudu/127.22.63.125@KRBTEST.COM (short username kudu)
I20260806 01:13:11.353395 25104 webserver.cc:460] Webserver started at http://127.22.63.125:34897/ using document root <none> and password file <none>
I20260806 01:13:11.354122 25104 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:11.354264 25104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:11.354611 25104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:11.357848 25104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data/instance:
uuid: "be7adf149dd841b7935e96a3054df331"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "cd663a9574ae6d2ef68e83d4fe902fb3"
server_key_iv: "34ab9ca9068f6752ce1d5193872ecb73"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.358668 25104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal/instance:
uuid: "be7adf149dd841b7935e96a3054df331"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "cd663a9574ae6d2ef68e83d4fe902fb3"
server_key_iv: "34ab9ca9068f6752ce1d5193872ecb73"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.364106 25104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 01:13:11.367585 25132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:11.368986 25104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 01:13:11.369181 25104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal
uuid: "be7adf149dd841b7935e96a3054df331"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "cd663a9574ae6d2ef68e83d4fe902fb3"
server_key_iv: "34ab9ca9068f6752ce1d5193872ecb73"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.369419 25104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:11.382462 25104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:11.386664 25104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:11.386981 25104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:11.405681 25104 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.125:35827
I20260806 01:13:11.405699 25208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.125:35827 every 8 connection(s)
I20260806 01:13:11.407073 25104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/data/info.pb
I20260806 01:13:11.410837 25209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:11.416988 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25104
I20260806 01:13:11.417227 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-1/wal/instance
I20260806 01:13:11.417797 22781 external_mini_cluster.cc:1428] Setting key e74c10bf5e844704dca4a9fed4ba0599
I20260806 01:13:11.417629 25209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:11 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:11.443099 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.420888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.22.63.126:37241)
0806 01:13:11.421442 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:11.421449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:11.422478 (+ 1029us) server_negotiation.cc:407] Connection header received
0806 01:13:11.425344 (+ 2866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:11.425348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:11.425423 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:11.425536 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:11.427683 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.428505 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.430900 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.431309 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.431420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.435847 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:11.435858 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:11.435866 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:11.435931 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:11.439764 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.440491 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.440500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.440506 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.440657 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.441063 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.441071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.441077 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.441476 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:11.441634 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:11.442321 (+ 687us) server_negotiation.cc:299] Negotiation successful
0806 01:13:11.442573 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":279,"threads_started":1}
I20260806 01:13:11.443910 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.421448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37241 (local address 127.0.0.1:58324)
0806 01:13:11.422282 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.422341 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:11.425027 (+ 2686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:11.425798 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:11.425812 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:11.426615 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:11.427435 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.427464 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.428676 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.428684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.430724 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.430743 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.431557 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.431567 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.431917 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.432705 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:11.432745 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:11.435569 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:11.439899 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.439906 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.439918 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.440362 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.440769 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.440772 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.440774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.440933 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.441602 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:11.441608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:11.441938 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 01:13:11.442190 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 01:13:11.442682 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":315,"threads_started":1}
I20260806 01:13:11.445611 25209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:11 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.63.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.63.124@KRBTEST.COM" created.
I20260806 01:13:11.462435 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.446517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.22.63.125:35827)
0806 01:13:11.447204 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:11.447214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:11.447248 (+ 34us) server_negotiation.cc:407] Connection header received
0806 01:13:11.447305 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:11.447316 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:11.447580 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:11.447727 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:11.448799 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.450490 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.451605 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.452589 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.452746 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.456306 (+ 3560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:11.456320 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:11.456337 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:11.456386 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:11.459451 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.460214 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.460221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.460223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.460339 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.460764 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.460771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.460778 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.461221 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:11.461382 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:11.461871 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 01:13:11.462123 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":487,"spinlock_wait_cycles":19712,"thread_start_us":316,"threads_started":1}
I20260806 01:13:11.462509 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.446445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58056)
0806 01:13:11.446635 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.446656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:11.446804 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:11.447803 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:11.447810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:11.448448 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:11.448622 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.448631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.450627 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.450630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.451455 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.451466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.452726 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.452730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.452848 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.453473 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:11.453501 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:11.456082 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:11.459572 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.459576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.459579 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.460067 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.460434 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.460437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.460439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.460621 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.461353 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:11.461358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:11.461534 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 01:13:11.461683 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 01:13:11.462006 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 01:13:11.464803 25209 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:11.466064 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.465517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37746)
0806 01:13:11.465726 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.465869 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 01:13:11.466362 25137 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.468055 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.467615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37758)
0806 01:13:11.467811 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.467914 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 01:13:11.468295 25209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
Entry for principal kudu/127.22.63.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab.
I20260806 01:13:11.503870 22781 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
/tmp/dist-test-taskfkFK3E/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:36349
--webserver_interface=127.22.63.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.63.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 01:13:11.517629 25209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 1
I20260806 01:13:11.518810 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.518371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37762)
0806 01:13:11.518559 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.518659 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 01:13:11.519874 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.519479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37770)
0806 01:13:11.519644 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.519709 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 01:13:11.520077 25209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.582562 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } attempt: 5
I20260806 01:13:11.588289 25209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 2
I20260806 01:13:11.589251 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.588838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37774)
0806 01:13:11.589010 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.589096 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 01:13:11.590196 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.589873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37782)
0806 01:13:11.590016 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.590090 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 01:13:11.590456 25209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Aug 06 01:13:11 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:11.597069 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.583695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.22.63.125:35827)
0806 01:13:11.583947 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:11.583953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:11.583971 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:13:11.584023 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:11.584026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:11.584088 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:11.584202 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:11.585046 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.585760 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.586940 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.587401 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.587550 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.591570 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:11.591575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:11.591578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:11.591646 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:11.594293 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.595271 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.595275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.595277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.595398 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:11.595836 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:11.595839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:11.595841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:11.596119 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:11.596233 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:11.596710 (+ 477us) server_negotiation.cc:299] Negotiation successful
0806 01:13:11.596877 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 01:13:11.597373 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.583375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58060)
0806 01:13:11.583581 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.583604 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:11.583724 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:11.584253 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:11.584259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:11.584668 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:11.584865 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.584874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.585906 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.585913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.586787 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:11.586799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.587572 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:11.587578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:11.587874 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:11.588568 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:11.588595 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:11.591381 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:11.594451 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.594458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.594465 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.595120 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.595521 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:11.595525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:11.595529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:11.595702 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:11.596254 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:11.596259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:11.596433 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 01:13:11.596599 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 01:13:11.596929 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 01:13:11.598449 25094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:11.599606 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.599153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37786)
0806 01:13:11.599341 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.599436 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 01:13:11.600682 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.600314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37798)
0806 01:13:11.600477 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.600559 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 01:13:11.600920 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.652173 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 1
I20260806 01:13:11.653398 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.652908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37800)
0806 01:13:11.653115 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.653211 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 01:13:11.654448 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.654136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37816)
0806 01:13:11.654272 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.654348 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 01:13:11.654760 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.681701 25209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 3
I20260806 01:13:11.682868 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.682380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37826)
0806 01:13:11.682610 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.682705 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 01:13:11.683977 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.683510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37836)
0806 01:13:11.683783 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.683861 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 01:13:11.684199 25209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.723011 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 2
I20260806 01:13:11.724437 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.723862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37848)
0806 01:13:11.724155 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.724248 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 01:13:11.725485 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.725105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37864)
0806 01:13:11.725293 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.725389 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 01:13:11.725703 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.827418 25209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 4
I20260806 01:13:11.828848 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.828212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37876)
0806 01:13:11.828606 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.828688 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
I20260806 01:13:11.829793 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.829483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37888)
0806 01:13:11.829624 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.829695 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 01:13:11.830006 25209 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
I20260806 01:13:11.831923 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 3
I20260806 01:13:11.832939 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.832547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37890)
0806 01:13:11.832716 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.832790 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 01:13:11.833871 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.833524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37904)
0806 01:13:11.833700 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:11.833778 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260806 01:13:11.834069 25094 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.63.124:36349: Network error: Client connection negotiation failed: client connection to 127.22.63.124:36349: connect: Connection refused (error 111)
W20260806 01:13:11.862185 25222 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 01:13:11.862587 25222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 01:13:11.862718 25222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 01:13:11.862864 25222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 01:13:11.870349 25222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 01:13:11.870473 25222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 01:13:11.870584 25222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 01:13:11.870678 25222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 01:13:11.870771 25222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 01:13:11.884965 25222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.63.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.63.126:37241,127.22.63.125:35827,127.22.63.124:36349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/krb5kdc/kudu_127.22.63.124.keytab
--principal=kudu/127.22.63.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.63.124:36349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.63.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 00:08:49 UTC on ccycloud.quasar-hjblmi.root.comops.site
build id 4569413
ASAN enabled
I20260806 01:13:11.886590 25222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 01:13:11.888427 25222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 01:13:11.903992 25232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:11.904105 25234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 01:13:11.904492 25231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 01:13:11.911988 25222 server_base.cc:1034] running on GCE node
I20260806 01:13:11.913307 25222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 01:13:11.914897 25222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 01:13:11.916153 25222 hybrid_clock.cc:648] HybridClock initialized: now 1785978791916094 us; error 38 us; skew 500 ppm
I20260806 01:13:11.916513 25222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 01:13:11 dist-test-slave-7f01 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 01:13:11.920178 25222 init.cc:377] Logged in from keytab as kudu/127.22.63.124@KRBTEST.COM (short username kudu)
I20260806 01:13:11.921936 25222 webserver.cc:460] Webserver started at http://127.22.63.124:33227/ using document root <none> and password file <none>
I20260806 01:13:11.922716 25222 fs_manager.cc:359] Metadata directory not provided
I20260806 01:13:11.922823 25222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 01:13:11.923131 25222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 01:13:11.926514 25222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data/instance:
uuid: "ef96ffbdb65641409d68657f322cca8b"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "3261969df55ca706edfe53e3961225ae"
server_key_iv: "e8c4cf8469c6352dac51629d9ae06c2f"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.927407 25222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal/instance:
uuid: "ef96ffbdb65641409d68657f322cca8b"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "3261969df55ca706edfe53e3961225ae"
server_key_iv: "e8c4cf8469c6352dac51629d9ae06c2f"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.932937 25222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 01:13:11.936630 25241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 01:13:11.937994 25222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 01:13:11.938187 25222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data,/tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal
uuid: "ef96ffbdb65641409d68657f322cca8b"
format_stamp: "Formatted at 2026-08-06 01:13:11 on dist-test-slave-7f01"
server_key: "3261969df55ca706edfe53e3961225ae"
server_key_iv: "e8c4cf8469c6352dac51629d9ae06c2f"
server_key_version: "encryptionkey@0"
I20260806 01:13:11.938437 25222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 01:13:11.957216 25222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 01:13:11.962196 25222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 01:13:11.962515 25222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 01:13:11.982740 25222 rpc_server.cc:307] RPC server started. Bound to: 127.22.63.124:36349
I20260806 01:13:11.982771 25314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.63.124:36349 every 8 connection(s)
I20260806 01:13:11.984216 25222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/data/info.pb
I20260806 01:13:11.987883 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 01:13:11.989902 22781 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu as pid 25222
I20260806 01:13:11.990110 22781 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfkFK3E/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785978402713339-22781-0/minicluster-data/master-2/wal/instance
I20260806 01:13:11.990600 22781 external_mini_cluster.cc:1428] Setting key 184bbcb7df768d2cc7d479c9bc380f84
I20260806 01:13:11.998088 25316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:12.011286 25094 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 4
I20260806 01:13:12.016578 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:11.992501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.22.63.124:36349)
0806 01:13:11.993369 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:11.993381 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:11.993422 (+ 41us) server_negotiation.cc:407] Connection header received
0806 01:13:11.995161 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:11.995184 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:11.995595 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:11.996042 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:11.997653 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:11.999642 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.001103 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.002281 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.002551 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.007107 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.007114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.007125 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.007169 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.011156 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.012122 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.012132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.012140 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.012341 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.012835 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.012841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.012845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.013566 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.013881 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.014996 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.015435 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":1120,"threads_started":1}
I20260806 01:13:12.019554 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.001300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.126:37241 (local address 127.0.0.1:58326)
0806 01:13:12.002142 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:12.002182 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:12.002383 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:12.002892 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:12.002901 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:12.003888 (+ 987us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:12.004289 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.004305 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.005574 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.005588 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.006754 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.006781 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.007560 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.007568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.007834 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.008415 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:12.008448 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:12.011429 (+ 2981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:12.016328 (+ 4899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.016341 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.016350 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.017001 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.017498 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.017503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.017506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.017716 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.018415 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:12.018422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:12.018612 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 01:13:12.018813 (+ 201us) client_negotiation.cc:240] Negotiation successful
0806 01:13:12.019153 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":454,"threads_started":1}
I20260806 01:13:12.020028 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.001232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.22.63.126:37241)
0806 01:13:12.001767 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.001777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.002300 (+ 523us) server_negotiation.cc:407] Connection header received
0806 01:13:12.002573 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.002579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.002664 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.002786 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.004454 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.005400 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.006941 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.007365 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.007458 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.011719 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.011726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.011752 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.011818 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.016145 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.017131 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.017142 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.017145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.017343 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.017851 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.017859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.017864 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.018260 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.018445 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.019236 (+ 791us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.019518 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":282,"threads_started":1}
I20260806 01:13:12.021612 25316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.126@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:12.029799 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.012038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37924)
0806 01:13:12.012223 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:12.012245 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:12.012341 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:12.012986 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:12.012994 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:12.013677 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:12.013926 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.013942 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.015522 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.015534 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.017317 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.017331 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.018255 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.018261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.018510 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.019092 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:12.019125 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:12.022863 (+ 3738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:12.026939 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.026948 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.026956 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.027528 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.028003 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.028007 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.028011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.028191 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.028852 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:12.028858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:12.029028 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 01:13:12.029192 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 01:13:12.029526 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.126@KRBTEST.COM
I20260806 01:13:12.030206 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.012066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.22.63.124:36349)
0806 01:13:12.012707 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.012711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.012730 (+ 19us) server_negotiation.cc:407] Connection header received
0806 01:13:12.012766 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.012770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.012830 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.012956 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.014109 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.015367 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.017461 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.018062 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.018808 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.023066 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.023073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.023077 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.023114 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.026749 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.027665 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.027673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.027676 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.027867 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.028344 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.028355 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.028358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.028699 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.028878 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.029289 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.029460 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":351,"threads_started":1}
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:12.035801 25094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: Bootstrap starting.
I20260806 01:13:12.039848 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.020612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.22.63.126:37241)
0806 01:13:12.020897 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.020905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.020923 (+ 18us) server_negotiation.cc:407] Connection header received
0806 01:13:12.020971 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.020979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.021061 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.021168 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.022688 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.023448 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.025351 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.026086 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.026241 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.031535 (+ 5294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.031542 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.031546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.031588 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.035818 (+ 4230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.036723 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.036729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.036732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.036875 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.037329 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.037334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.037337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.037741 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.038058 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.039280 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.039516 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260806 01:13:12.040259 25094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:12.041293 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.022373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.125:35827 (local address 127.0.0.1:58072)
0806 01:13:12.022544 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:12.022565 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:12.022668 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:12.023500 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:12.023508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:12.024087 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:12.024341 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.024355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.025777 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.025782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.027004 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.027021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.028161 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.028168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.028372 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.030255 (+ 1883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:12.030286 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:12.032669 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:12.037580 (+ 4911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.037589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.037593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.038288 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.038874 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.038881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.038887 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.039092 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.040308 (+ 1216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:12.040313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:12.040486 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 01:13:12.040654 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 01:13:12.040961 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 01:13:12.041761 25094 log.cc:824] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:12.042246 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.022777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.22.63.125:35827)
0806 01:13:12.022989 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.022995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.023015 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:13:12.023082 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.023087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.023173 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.023287 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.024517 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.025634 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.027182 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.027988 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.028133 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.032915 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.032923 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.032929 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.032976 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.037379 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.038456 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.038465 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.038471 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.038673 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.039249 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.039257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.039263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.039684 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.039890 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.040879 (+ 989us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.041260 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:13:12.043963 25316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
I20260806 01:13:12.046355 25094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: No bootstrap required, opened a new log
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.124@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:12.056754 25094 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.057129 25094 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8eea8b8c93749b8b92576fb64692ef7, State: Initialized, Role: FOLLOWER
I20260806 01:13:12.057715 25094 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.059202 25330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.059636 25330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.060269 25094 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:12.061828 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.045299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.22.63.124:36349)
0806 01:13:12.045487 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.045493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.045513 (+ 20us) server_negotiation.cc:407] Connection header received
0806 01:13:12.045566 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.045570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.045677 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.045786 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.047030 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.048506 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.050240 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.051046 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.051192 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.055243 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.055249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.055260 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.055290 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.058586 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.059403 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.059410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.059415 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.059557 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.060046 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.060051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.060055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.060494 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.060634 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.061201 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.061480 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 01:13:12.062711 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.044940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37926)
0806 01:13:12.045301 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:12.045321 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:12.045422 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:12.045976 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:12.045982 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:12.046571 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:12.046855 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.046872 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.048670 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.048677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.050077 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.050092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.051237 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.051243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.051476 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.052452 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:12.052477 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:12.055061 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:12.058737 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.058744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.058748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.059267 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.059707 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.059713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.059719 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.059922 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.060731 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:12.060735 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:12.060904 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 01:13:12.062018 (+ 1114us) client_negotiation.cc:240] Negotiation successful
0806 01:13:12.062339 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243}
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.63.125@KRBTEST.COM
I20260806 01:13:12.069844 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: Bootstrap starting.
I20260806 01:13:12.070138 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.052429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.22.63.125:35827)
0806 01:13:12.052621 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.052628 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.052645 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:13:12.052694 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.052700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.052778 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.052889 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.054364 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.055484 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.057418 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.058085 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.058229 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.064137 (+ 5908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.064143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.064149 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.064194 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.066856 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.067565 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.067584 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.067588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.067825 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.068381 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.068388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.068393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.068705 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.068890 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.069433 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.069724 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 01:13:12.074406 25316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:12.075887 25316 log.cc:824] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: Log is configured to *not* fsync() on all Append() calls
W20260806 01:13:12.077934 25350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 01:13:12.078074 25350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 01:13:12.079873 22781 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 01:13:12.081458 25316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: No bootstrap required, opened a new log
I20260806 01:13:12.082947 25057 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a8eea8b8c93749b8b92576fb64692ef7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58336
I20260806 01:13:12.083408 25057 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 01:13:12.083639 25057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:12.076354 24988 master_runner.cc:186] Error getting master registration for 127.22.63.125:35827: OK, instance_id { permanent_uuid: "be7adf149dd841b7935e96a3054df331" instance_seqno: 1785978791400399 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:13:12.086817 24988 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:13:12.087332 25057 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.088894 25057 leader_election.cc:288] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [CANDIDATE]: Term 1 election: Requested vote from peers be7adf149dd841b7935e96a3054df331 (127.22.63.125:35827), ef96ffbdb65641409d68657f322cca8b (127.22.63.124:36349)
I20260806 01:13:12.094156 25316 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.094662 25316 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef96ffbdb65641409d68657f322cca8b, State: Initialized, Role: FOLLOWER
I20260806 01:13:12.095301 25316 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.102353 25355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.102819 25355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.105042 25316 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:12.105119 25280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8eea8b8c93749b8b92576fb64692ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ef96ffbdb65641409d68657f322cca8b"
I20260806 01:13:12.105626 25280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:12.109308 25280 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8eea8b8c93749b8b92576fb64692ef7 in term 1.
I20260806 01:13:12.110121 25018 leader_election.cc:302] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8eea8b8c93749b8b92576fb64692ef7, ef96ffbdb65641409d68657f322cca8b; no voters:
I20260806 01:13:12.110656 25330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 01:13:12.111193 25025 tablet.cc:2369] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:12.124433 25330 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [term 1 LEADER]: Becoming Leader. State: Replica: a8eea8b8c93749b8b92576fb64692ef7, State: Running, Role: LEADER
I20260806 01:13:12.125180 25330 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.129191 25209 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } attempt: 5
Aug 06 01:13:12 dist-test-slave-7f01 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785978791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.63.125@KRBTEST.COM for kudu/127.22.63.124@KRBTEST.COM
I20260806 01:13:12.142859 25360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8eea8b8c93749b8b92576fb64692ef7. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.143363 25360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:12.148597 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.129768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.63.124:36349 (local address 127.0.0.1:37938)
0806 01:13:12.130160 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 01:13:12.130181 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 01:13:12.130282 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 01:13:12.131135 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 01:13:12.131142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 01:13:12.131687 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 01:13:12.131943 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.131957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.133496 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.133504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.135223 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 01:13:12.135242 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.136621 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.136629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 01:13:12.136850 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.137522 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 01:13:12.137552 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 01:13:12.141048 (+ 3496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 01:13:12.145442 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.145450 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.145455 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.146003 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.146490 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 01:13:12.146495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 01:13:12.146498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 01:13:12.146662 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 01:13:12.147462 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 01:13:12.147470 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 01:13:12.147672 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 01:13:12.147855 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 01:13:12.148157 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260806 01:13:12.149591 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.130504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.22.63.124:36349)
0806 01:13:12.130694 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.130700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.130716 (+ 16us) server_negotiation.cc:407] Connection header received
0806 01:13:12.130759 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.130764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.130830 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.130933 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.132133 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.133310 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.135421 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.136239 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.136379 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.141245 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.141251 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.141255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.141299 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.145275 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.146156 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.146162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.146165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.146310 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.146818 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.146822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.146825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.147225 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.147391 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.148986 (+ 1595us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.149228 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":45}
I20260806 01:13:12.156010 25209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: Bootstrap starting.
I20260806 01:13:12.156565 25366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 01:13:12.158241 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.111682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.22.63.125:35827)
0806 01:13:12.111888 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.111894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.120152 (+ 8258us) server_negotiation.cc:407] Connection header received
0806 01:13:12.120491 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.120500 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.120610 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.120694 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.123583 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.124723 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.126854 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.127499 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.127595 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.150512 (+ 22917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.150522 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.150530 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.150583 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.154776 (+ 4193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.155655 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.155664 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.155669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.155888 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.156388 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.156392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.156396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.156806 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.157043 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.157643 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.157906 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 01:13:12.161293 25366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 01:13:12.163375 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.118710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.22.63.126:37241)
0806 01:13:12.118905 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.118910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.127019 (+ 8109us) server_negotiation.cc:407] Connection header received
0806 01:13:12.127439 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.127446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.127536 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.127612 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.131396 (+ 3784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.132605 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.135628 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.136364 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.136473 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.142313 (+ 5840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.142321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.142325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.142366 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.146205 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.147662 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.147668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.147671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.147836 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.148569 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.148574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.148578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.148972 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.149294 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.162692 (+ 13398us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.162975 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:13:12.166579 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.111208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.22.63.124:36349)
0806 01:13:12.111528 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.111535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.124303 (+ 12768us) server_negotiation.cc:407] Connection header received
0806 01:13:12.128533 (+ 4230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.128540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.128622 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.128679 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.136390 (+ 7711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.137376 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.147956 (+ 10580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.148641 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.148747 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.158962 (+ 10215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 01:13:12.158968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 01:13:12.158983 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 01:13:12.159029 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 01:13:12.162609 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.163451 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.163456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.163459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.163642 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 01:13:12.164221 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 01:13:12.164226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 01:13:12.164229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 01:13:12.164646 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 01:13:12.164990 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.165745 (+ 755us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.166049 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260806 01:13:12.169793 25209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: Neither blocks nor log segments found. Creating new log.
I20260806 01:13:12.171996 25209 log.cc:824] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: Log is configured to *not* fsync() on all Append() calls
I20260806 01:13:12.174907 25280 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [term 1 FOLLOWER]: Refusing update from remote peer a8eea8b8c93749b8b92576fb64692ef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:12.176002 25360 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:12.187539 25209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: No bootstrap required, opened a new log
I20260806 01:13:12.203224 25209 raft_consensus.cc:348] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.203799 25209 raft_consensus.cc:729] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7adf149dd841b7935e96a3054df331, State: Initialized, Role: FOLLOWER
I20260806 01:13:12.204443 25209 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } }
I20260806 01:13:12.213595 25355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8eea8b8c93749b8b92576fb64692ef7. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.213938 25355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 01:13:12.214704 25250 tablet.cc:2369] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 01:13:12.223881 25370 mvcc.cc:205] Tried to move back new op lower bound from 7315369132733620224 to 7315369132550737920. Current Snapshot: MvccSnapshot[applied={T|T < 7315369132733620224}]
I20260806 01:13:12.223649 25380 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.225457 25380 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.226064 25375 mvcc.cc:205] Tried to move back new op lower bound from 7315369132733620224 to 7315369132550737920. Current Snapshot: MvccSnapshot[applied={T|T < 7315369132733620224}]
I20260806 01:13:12.230019 25173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a8eea8b8c93749b8b92576fb64692ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "be7adf149dd841b7935e96a3054df331"
I20260806 01:13:12.230818 25355 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.231056 25355 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.231890 25172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [term 0 FOLLOWER]: Advancing to term 1
I20260806 01:13:12.233547 25330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.234054 25209 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 01:13:12.234225 25330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:12.235754 25172 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [term 1 FOLLOWER]: Refusing update from remote peer a8eea8b8c93749b8b92576fb64692ef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 01:13:12.240296 25173 raft_consensus.cc:2454] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a8eea8b8c93749b8b92576fb64692ef7 in term 1.
I20260806 01:13:12.244392 25330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 01:13:12.244971 25366 catalog_manager.cc:1347] Generated new cluster ID: 9552638c742341128d1576a9aec4b0b4
I20260806 01:13:12.245069 25366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 01:13:12.258051 25389 mvcc.cc:205] Tried to move back new op lower bound from 7315369132733620224 to 7315369132550737920. Current Snapshot: MvccSnapshot[applied={T|T < 7315369132733620224}]
I20260806 01:13:12.261160 25380 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8eea8b8c93749b8b92576fb64692ef7. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.261482 25380 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.261873 25397 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.262387 25397 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 01:13:12.268882 25366 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 01:13:12.273774 25366 catalog_manager.cc:1504] Loading token signing keys...
I20260806 01:13:12.283874 25330 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a8eea8b8c93749b8b92576fb64692ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8eea8b8c93749b8b92576fb64692ef7" member_type: VOTER last_known_addr { host: "127.22.63.126" port: 37241 } } peers { permanent_uuid: "be7adf149dd841b7935e96a3054df331" member_type: VOTER last_known_addr { host: "127.22.63.125" port: 35827 } } peers { permanent_uuid: "ef96ffbdb65641409d68657f322cca8b" member_type: VOTER last_known_addr { host: "127.22.63.124" port: 36349 } } }
I20260806 01:13:12.284205 25330 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7 [sys.catalog]: This master's current role is: LEADER
I20260806 01:13:12.287431 25407 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 01:13:12.299269 25366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8eea8b8c93749b8b92576fb64692ef7: Generated new TSK 0
I20260806 01:13:12.300279 25366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 01:13:12.302248 25222 master_runner.cc:186] Error getting master registration for 127.22.63.125:35827: OK, instance_id { permanent_uuid: "be7adf149dd841b7935e96a3054df331" instance_seqno: 1785978791400399 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 01:13:12.331874 25222 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 01:13:12.335152 25423 catalog_manager.cc:1259] Loaded cluster ID: 9552638c742341128d1576a9aec4b0b4
I20260806 01:13:12.335287 25423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: loading cluster ID for follower catalog manager: success
I20260806 01:13:12.339476 25423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: acquiring CA information for follower catalog manager: success
I20260806 01:13:12.342347 25423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be7adf149dd841b7935e96a3054df331: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 01:13:12.402289 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.390183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.22.63.126:37241)
0806 01:13:12.390451 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.390457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.400295 (+ 9838us) server_negotiation.cc:407] Connection header received
0806 01:13:12.400631 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.400637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.400724 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.400799 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.402037 (+ 1238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58356: BlockingRecv error: recv got EOF from 127.0.0.1:58356 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 01:13:12.403662 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.392381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.22.63.124:36349)
0806 01:13:12.392548 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.392554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.400753 (+ 8199us) server_negotiation.cc:407] Connection header received
0806 01:13:12.401673 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.401679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.401769 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.401844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.403415 (+ 1571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37952: BlockingRecv error: recv got EOF from 127.0.0.1:37952 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 01:13:12.404690 25215 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.392557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.22.63.125:35827)
0806 01:13:12.392803 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.392811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.401548 (+ 8737us) server_negotiation.cc:407] Connection header received
0806 01:13:12.403053 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.403059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.403143 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.403241 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.404472 (+ 1231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58108: BlockingRecv error: recv got EOF from 127.0.0.1:58108 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 01:13:12.451607 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.441114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.22.63.125:35827)
0806 01:13:12.441303 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.441310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.444349 (+ 3039us) server_negotiation.cc:407] Connection header received
0806 01:13:12.444622 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.444628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.444812 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.444973 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.445532 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.446745 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.448818 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.449530 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.449664 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.450231 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.450880 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.451037 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.451171 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.451268 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 01:13:12.461179 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.458593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.22.63.124:36349)
0806 01:13:12.458830 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.458837 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.459358 (+ 521us) server_negotiation.cc:407] Connection header received
0806 01:13:12.459714 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.459720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.459881 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.459996 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.460939 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37964: BlockingRecv error: recv got EOF from 127.0.0.1:37964 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 01:13:12.478943 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.469859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.22.63.125:35827)
0806 01:13:12.470045 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.470050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.472292 (+ 2242us) server_negotiation.cc:407] Connection header received
0806 01:13:12.472652 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.472664 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.472780 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.472907 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.473432 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.474592 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.476692 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.477442 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.477587 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.478065 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.478232 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.478339 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.478515 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.478604 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 01:13:12.489639 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.487279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.22.63.124:36349)
0806 01:13:12.487461 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.487467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.488018 (+ 551us) server_negotiation.cc:407] Connection header received
0806 01:13:12.488214 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.488221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.488338 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.488455 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.489391 (+ 936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37968: BlockingRecv error: recv got EOF from 127.0.0.1:37968 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 01:13:12.503999 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.496404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58128 (local address 127.22.63.125:35827)
0806 01:13:12.496596 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.496602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.497114 (+ 512us) server_negotiation.cc:407] Connection header received
0806 01:13:12.497278 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.497284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.497395 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.497499 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.498130 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.499430 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.501718 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.502456 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.502592 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.503114 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.503260 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.503355 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.503539 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.503625 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 01:13:12.522025 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.513903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.22.63.124:36349)
0806 01:13:12.514089 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.514095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.520397 (+ 6302us) server_negotiation.cc:407] Connection header received
0806 01:13:12.520605 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.520613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.520721 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.520846 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.521753 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37976: BlockingRecv error: recv got EOF from 127.0.0.1:37976 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 01:13:12.539894 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.532390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.22.63.125:35827)
0806 01:13:12.532593 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.532599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.533148 (+ 549us) server_negotiation.cc:407] Connection header received
0806 01:13:12.533349 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.533361 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.533489 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.533597 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.534129 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.535359 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.537466 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.538222 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.538391 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.538918 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.539071 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.539184 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.539410 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.539522 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 01:13:12.550685 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.548163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.22.63.124:36349)
0806 01:13:12.548505 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.548512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.549091 (+ 579us) server_negotiation.cc:407] Connection header received
0806 01:13:12.549308 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.549315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.549419 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.549537 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.550453 (+ 916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37980: BlockingRecv error: recv got EOF from 127.0.0.1:37980 (error 108)
Metrics: {"server-negotiator.queue_time_us":200}
I20260806 01:13:12.570391 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.562510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58150 (local address 127.22.63.125:35827)
0806 01:13:12.562722 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.562727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.563319 (+ 592us) server_negotiation.cc:407] Connection header received
0806 01:13:12.563554 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.563560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.563651 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.563806 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.564599 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.565770 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.567955 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.568724 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.568856 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.569536 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.569683 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.569775 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.569965 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.570051 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 01:13:12.580291 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.577856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.22.63.124:36349)
0806 01:13:12.578054 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.578061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.578601 (+ 540us) server_negotiation.cc:407] Connection header received
0806 01:13:12.578803 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.578810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.578907 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.579016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.579999 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37984: BlockingRecv error: recv got EOF from 127.0.0.1:37984 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 01:13:12.608419 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.601264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.22.63.125:35827)
0806 01:13:12.601460 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.601469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.602016 (+ 547us) server_negotiation.cc:407] Connection header received
0806 01:13:12.602215 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.602221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.602316 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.602417 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.602895 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.604165 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.606177 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.606869 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.606996 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.607556 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.607729 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.607851 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.608036 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.608130 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:13:12.617551 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.615091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.22.63.124:36349)
0806 01:13:12.615270 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.615276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.615963 (+ 687us) server_negotiation.cc:407] Connection header received
0806 01:13:12.616136 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.616142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.616236 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.616339 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.617318 (+ 979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37986: BlockingRecv error: recv got EOF from 127.0.0.1:37986 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 01:13:12.668257 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.661833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.22.63.125:35827)
0806 01:13:12.662122 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.662128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.662468 (+ 340us) server_negotiation.cc:407] Connection header received
0806 01:13:12.662691 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.662696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.662777 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.662869 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.663391 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.664206 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.666204 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.666685 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.666801 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.667358 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.667477 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.667562 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.667834 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.667906 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 01:13:12.676615 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.674600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.22.63.124:36349)
0806 01:13:12.674835 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.674842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.675142 (+ 300us) server_negotiation.cc:407] Connection header received
0806 01:13:12.675311 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.675318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.675433 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.675559 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.676352 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37992: BlockingRecv error: recv got EOF from 127.0.0.1:37992 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 01:13:12.752638 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.746555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.22.63.125:35827)
0806 01:13:12.746854 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.746864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.747483 (+ 619us) server_negotiation.cc:407] Connection header received
0806 01:13:12.747649 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.747656 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.747768 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.747881 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.748469 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.749515 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.750850 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.751318 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.751465 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.751902 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.752021 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.752120 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.752280 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.752361 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260806 01:13:12.760895 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.758788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.22.63.124:36349)
0806 01:13:12.759007 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.759012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.759330 (+ 318us) server_negotiation.cc:407] Connection header received
0806 01:13:12.759501 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.759505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.759577 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.759660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.760665 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38004: BlockingRecv error: recv got EOF from 127.0.0.1:38004 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 01:13:12.898387 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:12.892982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58200 (local address 127.22.63.125:35827)
0806 01:13:12.893223 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.893232 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.893608 (+ 376us) server_negotiation.cc:407] Connection header received
0806 01:13:12.893785 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.893789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.893862 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.893945 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:12.894486 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.895192 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.896655 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:12.897067 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:12.897173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:12.897618 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:12.897721 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:12.897790 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:12.898006 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 01:13:12.898072 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:13:12.906263 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:12.904242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38006 (local address 127.22.63.124:36349)
0806 01:13:12.904492 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:12.904499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:12.904777 (+ 278us) server_negotiation.cc:407] Connection header received
0806 01:13:12.904996 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:12.905002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:12.905086 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:12.905196 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:12.906017 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38006: BlockingRecv error: recv got EOF from 127.0.0.1:38006 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260806 01:13:13.172413 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.166624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58210 (local address 127.22.63.125:35827)
0806 01:13:13.166822 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.166827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.167152 (+ 325us) server_negotiation.cc:407] Connection header received
0806 01:13:13.167377 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.167382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.167486 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.167581 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.168091 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.168972 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.170651 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.171076 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.171224 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.171657 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.171781 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.171889 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.172060 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.172144 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 01:13:13.180368 25327 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 01:13:13.178473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38016 (local address 127.22.63.124:36349)
0806 01:13:13.178699 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.178705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.178881 (+ 176us) server_negotiation.cc:407] Connection header received
0806 01:13:13.179110 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.179115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.179206 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.179321 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 01:13:13.180181 (+ 860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38016: BlockingRecv error: recv got EOF from 127.0.0.1:38016 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 01:13:13.292348 25407 catalog_manager.cc:1259] Loaded cluster ID: 9552638c742341128d1576a9aec4b0b4
I20260806 01:13:13.292516 25407 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: loading cluster ID for follower catalog manager: success
I20260806 01:13:13.295214 25407 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: acquiring CA information for follower catalog manager: success
I20260806 01:13:13.297073 25407 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef96ffbdb65641409d68657f322cca8b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 01:13:13.702644 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.697483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58218 (local address 127.22.63.125:35827)
0806 01:13:13.698002 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.698007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.698119 (+ 112us) server_negotiation.cc:407] Connection header received
0806 01:13:13.698275 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.698279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.698362 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.698451 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.698934 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.699688 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.701017 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.701420 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.701517 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.701879 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.701982 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.702048 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.702182 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.702250 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":279,"threads_started":1}
I20260806 01:13:13.714485 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.708688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.22.63.124:36349)
0806 01:13:13.709166 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.709171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.709188 (+ 17us) server_negotiation.cc:407] Connection header received
0806 01:13:13.709314 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.709318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.709480 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.709583 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.709948 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.710684 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.712102 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.712501 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.712597 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.713093 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.713662 (+ 569us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.713788 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.713928 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.714006 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":269,"threads_started":1}
I20260806 01:13:13.727948 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.720538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.22.63.125:35827)
0806 01:13:13.720753 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.720762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.722194 (+ 1432us) server_negotiation.cc:407] Connection header received
0806 01:13:13.722553 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.722557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.722653 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.722748 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.723295 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.724355 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.726218 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.726590 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.726727 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.727175 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.727280 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.727375 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.727530 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.727607 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 01:13:13.729768 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.721386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.22.63.124:36349)
0806 01:13:13.721580 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.721589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.722657 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 01:13:13.722877 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.722882 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.722978 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.723039 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.723914 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.724629 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.727508 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.728159 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.728258 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.729089 (+ 831us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.729212 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.729307 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.729463 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.729523 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 01:13:13.730234 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 01:13:13.720504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.22.63.126:37241)
0806 01:13:13.721193 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 01:13:13.721201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 01:13:13.722050 (+ 849us) server_negotiation.cc:407] Connection header received
0806 01:13:13.722378 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 01:13:13.722386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 01:13:13.722621 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 01:13:13.722796 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 01:13:13.723708 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.724801 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.727511 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 01:13:13.728159 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 01:13:13.728274 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 01:13:13.729180 (+ 906us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 01:13:13.729620 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 01:13:13.729733 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 01:13:13.729878 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 01:13:13.729945 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":382,"threads_started":1}
I20260806 01:13:13.734380 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 24988
I20260806 01:13:13.735392 25088 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:13.847329 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 24988
I20260806 01:13:13.879261 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25104
I20260806 01:13:13.880596 25202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:13.990069 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25104
I20260806 01:13:14.018322 22781 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu pid 25222
I20260806 01:13:14.019614 25308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 01:13:14.126653 22781 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfkFK3E/build/asan/bin/kudu with pid 25222
2026-08-06T01:13:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5400 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10647 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (391380 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST